feat(io): uncached reads on macOS via F_NOCACHE, and o_direct from the open's outcome (#182)

A direct request on Apple opens normally and applies fcntl(F_NOCACHE, 1) to the descriptor instead of silently returning a buffered fd. F_NOCACHE is a caching hint, not an I/O mode: no alignment contract, no DMA promise, so a direct reader on Apple keeps plain pread semantics (pio::direct_needs_alignment() splits "uncached descriptor" from "alignment-constrained reads") and skips the O_DIRECT bounce path.

Independently, the o_direct telemetry (CSV preamble, decode-trace header, streaming banner) now reports what the shard opens achieved, the AND across shards after every platform refusal and open-time downgrade, instead of the requested configuration, on every platform.

Measured on a 16 GB Apple-silicon Mac, model on an external volume, 256-token protocol, interleaved A B B A A B: decode 0.94 vs 0.61 tok/s (+56%, non-overlapping), byte stream identical between arms, cache-hit equal; the gain is the buffered arm's page-cache pollution doubling the compute residual while stall stays flat.

Addresses the macOS half of #179.

CHANGELOG.md

@@ -83,6 +83,17 @@ Semantic Versioning.
 - `--cache-mb auto` no longer reserves the size of a row-streamed table for a conversion that will
   not happen. It deducts the window instead, so the cache gets the RAM the policy freed rather than
   the engine planning for both.
+- **macOS reads uncached (`F_NOCACHE`), and `o_direct` reports the open's real outcome everywhere.**
+  A direct request on Apple applies `fcntl(F_NOCACHE, 1)` to every reader descriptor — the kernel
+  stops caching that file's pages — instead of silently returning a buffered fd (Apple has no
+  `O_DIRECT`). It is a caching hint, not an I/O mode: no alignment contract, no DMA promise, so a
+  direct reader on Apple keeps ordinary `pread` semantics and skips the O_DIRECT bounce path rather
+  than inheriting Linux-only alignment requirements. Independently, the `o_direct` telemetry field
+  (CSV preamble, decode-trace header, streaming banner) now comes from what the shard opens
+  achieved — the AND across shards, after every platform refusal and open-time downgrade — instead
+  of from the requested configuration, so a run served buffered says `0` on every platform. Host
+  measurements in the PR (#179).
+
 ## [0.23.0] - 2026-08-29
 
 ### Fixed

README.md

@@ -401,8 +401,9 @@ The model must live on a real filesystem (on Android `/data/local/tmp/...`, not
 Platform status: Linux is exercised by CI (build + gates) and Windows is where the
 [desktop numbers](#desktop) were measured. On Windows, build with CMake directly (Visual Studio
 Build Tools); the script above is bash, and MSVC puts the binary in `build\cli\Release\bmoe-cli.exe`.
-macOS builds from the same sources (the platform branches exist) but is not validated, and it has
-no O_DIRECT, so direct reads fall back to buffered I/O there.
+macOS builds from the same sources (the platform branches exist) but is not exercised by CI. It has
+no O_DIRECT; a direct request is served with `F_NOCACHE` instead (uncached, but not alignment-constrained),
+and `o_direct` in the telemetry reports what the open actually achieved.
 
 ### Android
 

core/include/bmoe/expert_source.h

@@ -81,6 +81,11 @@ public:
         long long spec_useful = 0;         // prefetched experts that a later lookup actually hit
         uint64_t cache_budget_bytes = 0;   // cache budget in force; fixed for the run once init sizes it
         long long cache_resizes = 0;       // explicit set_cache_budget_mb() calls that moved the budget
+        // Whether cache-bypassing reads are actually in effect for the expert shards — every shard
+        // reader's request honoured by the platform (O_DIRECT open succeeded, F_NOCACHE applied on
+        // Apple), after the open-time downgrades. NOT the config flag: a run can ask for direct and
+        // be served buffered, and the telemetry must say which one it got.
+        bool o_direct = false;
         // Cache churn. `evictions` is how many entries the budget forced out; `rereads` how many
         // reads went to an entry that had been resident before — the cache paying for the same
         // bytes twice. A prefetch cannot reduce what a routing needs (the ideal is the same

core/src/engine/session.cpp

@@ -947,7 +947,7 @@ std::unique_ptr<Session> Session::open(const SessionConfig & cfg,
         st.n_layer = im.n_layer;
         st.n_threads = cfg.n_threads;
         st.io_threads = cfg.moe.enabled ? cfg.moe.io_threads : 0;
-        st.o_direct = cfg.moe.enabled && cfg.moe.o_direct;
+        st.o_direct = cfg.moe.enabled && im.source.stats().o_direct; // the open's outcome, not the request
         st.overlap = cfg.moe.enabled && cfg.moe.overlap;
         if (compute_trace) {
             im.compute_trace = compute_trace;
@@ -988,7 +988,6 @@ std::unique_ptr<Session> Session::open(const SessionConfig & cfg,
         ri.force_cache = cfg.moe.force_cache;
         ri.load_all = cfg.moe.enabled && cfg.moe.load_all;
         ri.io_threads = cfg.moe.enabled ? cfg.moe.io_threads : 0;
-        ri.o_direct = cfg.moe.enabled && cfg.moe.o_direct;
         ri.overlap = cfg.moe.enabled && cfg.moe.overlap;
         ri.io_two_wave = cfg.moe.enabled && cfg.moe.io_two_wave;
         ri.prefetch_layers = cfg.moe.enabled ? cfg.moe.prefetch_layers : 0;
@@ -1009,6 +1008,9 @@ std::unique_ptr<Session> Session::open(const SessionConfig & cfg,
         if (cfg.moe.enabled) {
             const IExpertSource::Stats st = im.source.stats();
             ri.cache_mb = (int) (st.cache_budget_bytes / (1024ull * 1024ull));
+            // o_direct from the same sample: whether the shard readers actually got cache bypass
+            // (O_DIRECT honoured, or F_NOCACHE applied on Apple), never what the flag asked for.
+            ri.o_direct = st.o_direct;
         }
         // The EFFECTIVE top-k: an override IS the applied width, otherwise the model's own. Same
         // resolution the route trace does, and worth a header read — a run whose top-k is unknown

core/src/io/file_reader.cpp

@@ -19,8 +19,8 @@ bool FileReader::open(const std::string & path, int lanes, bool direct, size_t a
     direct_ = direct;
     const int n = lanes < 1 ? 1 : lanes;
 
-    pio::fd_t primary = pio::open_read(path.c_str(), direct_);
-    if (!pio::fd_ok(primary) && direct_) { // the platform refused O_DIRECT outright — try buffered
+    pio::fd_t primary = pio::open_read(path.c_str(), direct, &direct_); // direct_ := the open's real outcome
+    if (!pio::fd_ok(primary) && direct) { // the platform refused the direct open outright — try buffered
         direct_ = false;
         primary = pio::open_read(path.c_str(), false);
     }
@@ -30,12 +30,17 @@ bool FileReader::open(const std::string & path, int lanes, bool direct, size_t a
     }
     fsize_ = pio::file_size(primary);
 
+    // The read mechanics follow the platform's direct MODE, not the direct FACT: O_DIRECT and
+    // FILE_FLAG_NO_BUFFERING reject unaligned windows and need the bounce and the buffered tail fd;
+    // Apple's F_NOCACHE is a caching hint, so an uncached reader there keeps plain preads.
+    aligned_reads_ = direct_ && pio::direct_needs_alignment();
+
     // Verify O_DIRECT actually returns correct bytes on this storage. On some emulated / FUSE-backed
     // volumes (e.g. an app-private dir under /storage/emulated) the open SUCCEEDS but direct reads
     // return garbage, silently corrupting weights → nonsense output. Compare one aligned block read
     // directly against the same block read buffered; on any mismatch, fall back to buffered I/O for
     // this file. On real filesystems (adb-pushed models, desktop) the two match and O_DIRECT is kept.
-    if (direct_ && fsize_ >= (uint64_t) align_) {
+    if (aligned_reads_ && fsize_ >= (uint64_t) align_) {
         uint64_t voff = (fsize_ / 2) & ~(uint64_t) (align_ - 1);
         if (voff + align_ > fsize_) voff = 0;
         void * dbuf = pio::alloc_aligned(align_, align_);
@@ -59,6 +64,7 @@ bool FileReader::open(const std::string & path, int lanes, bool direct, size_t a
             std::fprintf(stderr, "bmoe: reopen after O_DIRECT check failed\n");
             return false;
         }
+        aligned_reads_ = aligned_reads_ && direct_; // a verify downgrade reverts the mechanics too
     }
 
     // A private fd + bounce per lane so concurrent reads never contend. Lane 0 inherits the fd already
@@ -69,7 +75,7 @@ bool FileReader::open(const std::string & path, int lanes, bool direct, size_t a
     bounce_sz_.assign(n, 0);
     for (int lane = 0; lane < n; ++lane) {
         fds_[lane] = (lane == 0) ? primary : pio::open_read(path.c_str(), direct_);
-        fds_buf_[lane] = direct_ ? pio::open_read(path.c_str(), false) : pio::fd_invalid;
+        fds_buf_[lane] = aligned_reads_ ? pio::open_read(path.c_str(), false) : pio::fd_invalid;
         if (!pio::fd_ok(fds_[lane])) {
             std::fprintf(stderr, "bmoe: lane %d open failed\n", lane);
             close();
@@ -78,15 +84,17 @@ bool FileReader::open(const std::string & path, int lanes, bool direct, size_t a
         // Not fatal — the buffered fd is only needed for a read that runs into a sub-alignment EOF
         // tail, which expert slices never do. But say so here, where the cause (fd pressure) is still
         // visible, rather than leaving a later tail read to fail with an unexplained EINVAL.
-        if (direct_ && !pio::fd_ok(fds_buf_[lane]))
+        if (aligned_reads_ && !pio::fd_ok(fds_buf_[lane]))
             std::fprintf(stderr, "bmoe: lane %d buffered tail fd unavailable — EOF-tail reads will fail\n", lane);
-        bounces_[lane] = pio::alloc_aligned(align_, bounce_cap);
-        if (!bounces_[lane]) {
+        // No bounce where reads stay plain (Apple F_NOCACHE): a per-lane aligned reservation next to
+        // the expert cache would buy nothing there.
+        bounces_[lane] = aligned_reads_ ? pio::alloc_aligned(align_, bounce_cap) : nullptr;
+        if (aligned_reads_ && !bounces_[lane]) {
             std::fprintf(stderr, "bmoe: lane %d bounce alloc failed\n", lane);
             close();
             return false;
         }
-        bounce_sz_[lane] = bounce_cap;
+        bounce_sz_[lane] = aligned_reads_ ? bounce_cap : 0;
     }
     return true;
 }
@@ -95,12 +103,12 @@ long long FileReader::read(int lane, void * dst, uint64_t off, uint64_t nbytes)
     if (nbytes == 0) return 0;
     const pio::fd_t fd = fds_[lane];
 
-    // Buffered mode: no alignment constraint, so the bounce buys nothing. Read straight into the
-    // caller's memory. This is the fallback path (the platform refused O_DIRECT, or the open-time
-    // verify caught storage that mis-serves it) and is already the slow mode — it must not also pay
-    // an extra copy of every byte plus a leading partial-block over-read just to share the mechanics
-    // O_DIRECT needs.
-    if (!direct_) {
+    // Plain reads: no alignment constraint, so the bounce buys nothing. Read straight into the
+    // caller's memory. This is the path for any reader whose direct mode imposes no alignment — the
+    // fallback after a refused open or a failed open-time verify, and Apple's F_NOCACHE, which is
+    // uncached without being an I/O mode. It must not also pay an extra copy of every byte plus a
+    // leading partial-block over-read just to share the mechanics O_DIRECT needs.
+    if (!aligned_reads_) {
         const uint64_t end = (fsize_ && off + nbytes > fsize_) ? fsize_ : off + nbytes;
         const auto t0 = clock_t_::now();
         for (uint64_t a = off; a < end;) {

core/src/io/file_reader.h

@@ -1,16 +1,22 @@
-// A pooled, positioned file reader with optional cache-bypassing (O_DIRECT) I/O.
+// A pooled, positioned file reader with optional cache-bypassing I/O (O_DIRECT on POSIX,
+// FILE_FLAG_NO_BUFFERING on Windows, F_NOCACHE on Apple).
 //
 // One reader owns N lane fds and N bounce buffers, so N threads can read distinct byte ranges of the
-// same file concurrently without contending. A direct read aligns its window to the device block
-// size, pulls it into the lane's bounce buffer, and memcpy's the requested interior out — the
-// mechanics O_DIRECT requires; a buffered read has no such constraint and goes straight into the
-// caller's memory. `direct` is a property of the reader, chosen by whoever opens it: the expert
-// streamer and the dense-weights loader each construct their own, so one can bypass the page cache
-// while the other does not — the two O_DIRECT decisions are independent, not a shared global.
+// same file concurrently without contending. Where the platform's direct mode is an I/O mode that
+// rejects unaligned access (O_DIRECT, NO_BUFFERING), a direct read aligns its window to the read
+// alignment, pulls it into the lane's bounce buffer, and memcpy's the requested interior out — the
+// mechanics those modes require. Apple's F_NOCACHE is not such a mode — a caching hint on the
+// descriptor — so a direct reader there keeps ordinary pread semantics and skips the bounce
+// entirely; see pio::direct_needs_alignment(). `direct` is a property of the reader, chosen by
+// whoever opens it: the expert streamer and the dense-weights loader each construct their own, so
+// one can bypass the page cache while the other does not — the two decisions are independent, not
+// a shared global.
 //
 // The O_DIRECT request is verified once at open: on storage where a direct read returns garbage
 // (some FUSE-backed emulated volumes) the reader silently falls back to buffered I/O for the file,
 // so a caller never has to reason about the storage — it asks for direct and gets correct bytes.
+// direct() reports the effective mode after all of that, which is what telemetry must print, not
+// what the caller requested.
 #pragma once
 
 #include "platform_io.h"
@@ -30,22 +36,25 @@ public:
     FileReader & operator=(const FileReader &) = delete;
 
     // Open `path` with `lanes` independent primary fds (+ a buffered fd per lane for the sub-alignment
-    // EOF tail an O_DIRECT read cannot cover) and a `bounce_cap`-byte aligned bounce per lane. `direct`
-    // requests O_DIRECT; it is verified and silently downgraded on storage that mis-serves it. Reads
-    // align to `align`. Returns false on any open/alloc failure. A reader is opened once and not reused.
+    // EOF tail an alignment-constrained direct read cannot cover) and a `bounce_cap`-byte aligned
+    // bounce per lane. `direct` requests cache bypass; it is verified and silently downgraded where
+    // the platform or the storage refuses or mis-serves it — direct() then reports the effective
+    // mode. Reads align to `align` where the platform's direct mode demands it. Returns false on any
+    // open/alloc failure. A reader is opened once and not reused.
     bool open(const std::string & path, int lanes, bool direct, size_t align, size_t bounce_cap);
     void close();
 
     bool is_open() const { return !fds_.empty(); }
-    bool direct() const { return direct_; } // the effective mode after verification/fallback
+    bool direct() const { return direct_; } // the effective cache-bypass mode, not the request
     uint64_t file_size() const { return fsize_; }
     int lanes() const { return (int) fds_.size(); }
 
     // Read `nbytes` at file offset `off` into `dst`, on `lane` (0 <= lane < lanes()). Thread-safe
     // across distinct lanes — each has its own fd and bounce. Returns what was actually pulled from
-    // the drive — the aligned window (>= nbytes) when direct, exactly the requested bytes when
-    // buffered — which is what the bandwidth must be judged against; or -1 on I/O error. A
-    // zero-length read is a no-op returning 0. The lane's bounce grows if a direct read needs more.
+    // the drive — the aligned window (>= nbytes) where the platform's direct mode requires one,
+    // exactly the requested bytes otherwise — which is what the bandwidth must be judged against;
+    // or -1 on I/O error. A zero-length read is a no-op returning 0. The lane's bounce grows if a
+    // windowed read needs more.
     long long read(int lane, void * dst, uint64_t off, uint64_t nbytes);
 
     // Aggregate accounting since open, summed across lanes.
@@ -53,13 +62,16 @@ public:
     long long syscall_ns() const { return syscall_ns_.load(std::memory_order_relaxed); }
 
 private:
-    std::vector<pio::fd_t> fds_;     // primary (maybe O_DIRECT) per lane
+    std::vector<pio::fd_t> fds_;     // primary (cache-bypassing where achieved) per lane
     std::vector<pio::fd_t> fds_buf_; // buffered fallback per lane, for the sub-alignment EOF tail
     std::vector<void *> bounces_;
     std::vector<size_t> bounce_sz_;
     size_t align_ = 4096;
     uint64_t fsize_ = 0;
-    bool direct_ = false;
+    bool direct_ = false;        // cache bypass actually in effect for the fds (uncached descriptors),
+                                 // after the open's own report and every fallback
+    bool aligned_reads_ = false; // direct_ AND the platform's direct mode rejects unaligned reads —
+                                 // gates the read mechanics: window rounding, bounce, buffered tail fd
     std::atomic<long long> read_bytes_{0};
     std::atomic<long long> syscall_ns_{0};
 };

core/src/io/platform_io.cpp

@@ -51,9 +51,10 @@ bool fd_ok(fd_t fd) {
     return fd != (void *) INVALID_HANDLE_VALUE;
 }
 
-fd_t open_read(const char * path, bool direct) {
+fd_t open_read(const char * path, bool direct, bool * effective_direct) {
     DWORD flags = FILE_ATTRIBUTE_NORMAL | (direct ? FILE_FLAG_NO_BUFFERING : 0);
     HANDLE h = CreateFileA(path, GENERIC_READ, FILE_SHARE_READ, nullptr, OPEN_EXISTING, flags, nullptr);
+    if (effective_direct) *effective_direct = direct && fd_ok((fd_t) h);
     return (fd_t) h;
 }
 
@@ -166,8 +167,21 @@ bool fd_ok(fd_t fd) {
     return fd >= 0;
 }
 
-fd_t open_read(const char * path, bool direct) {
-    return open(path, O_RDONLY | O_CLOEXEC | (direct ? O_DIRECT : 0));
+fd_t open_read(const char * path, bool direct, bool * effective_direct) {
+#if defined(__APPLE__)
+    // No O_DIRECT here (the shim above leaves it 0), so a direct request is an ordinary open plus
+    // F_NOCACHE: ask the kernel not to keep this descriptor's pages. That is the whole of Apple's
+    // uncached mode — a caching hint on the fd, not an I/O mode — so a refusal is a downgrade to
+    // buffered for the caller to report, never a reason to fail a perfectly good descriptor.
+    const fd_t fd = open(path, O_RDONLY | O_CLOEXEC);
+    const bool ok = fd_ok(fd) && direct && fcntl(fd, F_NOCACHE, 1) == 0;
+    if (effective_direct) *effective_direct = ok;
+    return fd;
+#else
+    const fd_t fd = open(path, O_RDONLY | O_CLOEXEC | (direct ? O_DIRECT : 0));
+    if (effective_direct) *effective_direct = direct && fd_ok(fd);
+    return fd;
+#endif
 }
 
 void close_fd(fd_t fd) {
@@ -374,6 +388,15 @@ bool file_mapped_regions(const char * basename, std::vector<MappedRegion> & out)
 
 #endif
 
+// ── Cache-bypass read semantics ─────────────────────────────────────────────────────────
+bool direct_needs_alignment() {
+#if defined(__APPLE__)
+    return false; // F_NOCACHE is a caching hint, not an I/O mode: plain preads stay fully general
+#else
+    return true; // O_DIRECT / FILE_FLAG_NO_BUFFERING reject unaligned windows
+#endif
+}
+
 // ── Reclaim-exempt allocation ────────────────────────────────────────────────────────────
 // Shared across platforms because only Android has one: everywhere else this reports "unsupported"
 // and callers fall back to an ordinary allocation. Declared in the header with the measured

core/src/io/platform_io.h

@@ -30,14 +30,27 @@ using fd_t = int;
 extern const fd_t fd_invalid;
 bool fd_ok(fd_t fd);
 
-// Open path for positioned reads. When direct is true, request cache-bypassing I/O;
-// the caller should be prepared to reopen with direct=false for a sub-alignment tail.
-fd_t open_read(const char * path, bool direct);
+// Open path for positioned reads. When direct is true, request cache-bypassing I/O — O_DIRECT on
+// Linux/Android, FILE_FLAG_NO_BUFFERING on Windows, F_NOCACHE on Apple. The descriptor itself is
+// valid either way; *effective_direct (when not null) reports whether the cache-bypassing mode is
+// actually in effect for it, because every platform can decline the request without failing the
+// open — a caller that cares must ask here, never reconstruct the answer from what it passed.
+// The caller should be prepared to reopen with direct=false for a sub-alignment tail.
+fd_t open_read(const char * path, bool direct, bool * effective_direct = nullptr);
 void close_fd(fd_t fd);
 // Positioned blocking read. Returns bytes read, 0 at EOF, -1 on error.
 long long pread_at(fd_t fd, void * buf, size_t count, uint64_t off);
 uint64_t file_size(fd_t fd);
 
+// Whether the platform's cache-bypassing mode constrains the read mechanics: O_DIRECT and
+// FILE_FLAG_NO_BUFFERING reject unaligned offsets, lengths and buffers, so reads must go through
+// aligned windows into an aligned bounce. Apple's F_NOCACHE — the only uncached mode there, since
+// the platform has no O_DIRECT — is a per-descriptor hint to turn data caching off: no alignment
+// contract, no DMA promise. "Uncached" (a property of the descriptor, what open_read reports) and
+// "alignment-constrained" (a property of the I/O mode, what this reports) are the same thing on
+// every platform except Apple, where they come apart.
+bool direct_needs_alignment();
+
 // Aligned heap allocation for O_DIRECT bounce buffers and shared slots.
 void * alloc_aligned(size_t align, size_t sz);
 void aligned_free(void * p);

core/src/moe/expert_stream_source.cpp

@@ -205,6 +205,14 @@ bool ExpertStreamSource::init(const std::vector<std::string> & shard_paths,
         readers_.push_back(std::unique_ptr<FileReader>(new FileReader()));
         if (!readers_.back()->open(sp, io_threads_, cfg.o_direct, align_, bounce_cap)) return false;
     }
+    // Each shard resolved direct for itself at open — the platform's answer plus the open-time
+    // verify — so the run-level fact is the weakest shard: a mixed run must not be advertised as
+    // fully direct, and a metadata-only first shard that never carried an expert read must not
+    // flatter the shards that do. Settled here, before any worker thread exists; stats() reads it
+    // cross-thread for the rest of the run.
+    effective_direct_ = true;
+    for (const auto & r : readers_)
+        effective_direct_ = effective_direct_ && r->direct();
     for (const LayerExperts & L : layers_) {
         if (!L.bound) continue;
         for (int p = 0; p < MoeRecipe::max_exps; ++p)
@@ -278,13 +286,9 @@ bool ExpertStreamSource::init(const std::vector<std::string> & shard_paths,
     for (int lane = first_worker_lane; lane < io_threads_; ++lane)
         io_pool_.emplace_back(&ExpertStreamSource::io_worker, this, lane);
 
-    // Each shard verified O_DIRECT for itself, so report the weakest: a metadata-only first shard
-    // is too short to verify at all and would flatter the number for the shards carrying experts.
-    bool all_direct = true;
-    for (const auto & r : readers_)
-        all_direct = all_direct && r->direct();
+    // effective_direct_ (set where the readers opened) is the run's o_direct fact — see above.
     std::fprintf(stderr, "bmoe: expert streaming ON  n_expert=%d o_direct=%d io_threads=%d cache=%zu MiB shards=%zu\n",
-                 n_expert_, (int) all_direct, io_threads_, cache_max_ >> 20, readers_.size());
+                 n_expert_, (int) effective_direct_, io_threads_, cache_max_ >> 20, readers_.size());
     return true;
 }
 
@@ -1383,6 +1387,7 @@ IExpertSource::Stats ExpertStreamSource::stats() const {
     s.stall_seconds = stall_union_.total_ns() / 1e9;
     s.cache_budget_bytes = (uint64_t) cache_max_;
     s.cache_resizes = cache_resizes_;
+    s.o_direct = effective_direct_;
     s.evictions = evictions_;
     s.rereads = rereads_;
     s.drain_wait_seconds = drain_wait_ns_ / 1e9;

core/src/moe/expert_stream_source.h

@@ -247,6 +247,10 @@ private:
     // them; the dense-weights loader constructs its own, so their O_DIRECT choices are independent
     // (see docs/architecture.md). FileReader is not movable, hence the unique_ptr.
     std::vector<std::unique_ptr<FileReader>> readers_;
+    // Whether every shard reader actually got cache bypass (see FileReader::direct) — the AND over
+    // shards, settled at init before any worker thread exists and reported verbatim by stats(). The
+    // run-level o_direct fact: what the platform served, not what the flag asked for.
+    bool effective_direct_ = false;
 
     std::vector<LayerExperts> layers_;
 

docs/community-benchmarks.md

@@ -50,7 +50,7 @@ empty unless you run the storage probe yourself.
 
 | | Status |
 |---|---|
-| **Mac**, Apple silicon or Intel | Works, with a caveat. macOS has no `O_DIRECT` and the engine does not yet call the `F_NOCACHE` equivalent, so expert reads go through the page cache instead of bypassing it — and the run still prints `o_direct=1`, because that field records what was *asked for*. Say "macOS" in your report: the cache-hit and flash-per-token columns are not measuring the same thing as a Linux or Android row. A 64 or 128 GB Mac is still a row we want. |
+| **Mac**, Apple silicon or Intel | Works. A direct request is served with `F_NOCACHE` (Apple's uncached-descriptor mode — a caching hint, not Linux `O_DIRECT`: no alignment, no DMA promise), and `o_direct` reports what the open actually achieved. Uncached reads can sit below the buffered rate on the same drive, so a Mac row's flash column may read slower than the storage probe suggests; that is the honest number, not a malfunction. Say "macOS" in your report. A 64 or 128 GB Mac is still a row we want. |
 | **iPhone, iPad** | Not supported. No iOS target here, and iOS does not run command-line binaries, so reproducing the protocol means building an app around the engine and sideloading it. The core is portable C++ with no Android dependency in the streaming path, so a port is plausible, it just does not exist. |
 
 ### What happens to your row

docs/limitations.md

@@ -52,13 +52,17 @@ serial path, and only a single ~25-line hook (with an explicit sunset) for the o
   faster loaded resident, and the registry rows are about coverage, not a recommendation.
 - **Repack must stay off.** Loading uses `use_extra_bufts=false`; you cannot combine
   streaming with weight repacking.
-- **macOS does not bypass the page cache, and the telemetry does not say so.** Apple has no
-  `O_DIRECT`; `platform_io` compiles it away to `0`, and the `fcntl(F_NOCACHE)` equivalent is not
-  called, so expert reads on a Mac go through the page cache the whole design exists to avoid.
-  Worse, `o_direct` in the metrics records the requested configuration rather than what the open
-  actually did, so a macOS run reports `o_direct=1` while running buffered. macOS builds and
-  produces correct output; its cache-hit and flash-per-token columns are not comparable with a
-  Linux or Android row until this is fixed.
+- **macOS reads uncached, and `o_direct` says so — but it is not `O_DIRECT`.** A direct request on
+  Apple is served by `fcntl(F_NOCACHE)` on each descriptor: the kernel stops caching that file's
+  pages, which is the property the design wants. It is a caching hint, not an I/O mode — no
+  alignment contract, no DMA promise — so reads keep ordinary `pread` semantics and the raw-read
+  ceiling can sit below Linux `O_DIRECT`'s. Measured on one 16 GB Apple-silicon Mac with the model
+  on an external volume (256-token protocol, buffered vs `F_NOCACHE`, interleaved A B B A A B):
+  decode 0.94 vs 0.61 tok/s (+56 %, arm ranges non-overlapping), with the byte stream
+  bit-identical between arms (40 822.8 MiB read by both) and cache-hit equal — the gain is not
+  fewer reads but the buffered arm's page-cache pollution doubling the compute residual
+  (1.22 → 0.60 s/tok) while stall stays flat (0.43 → 0.46 s/tok). The `o_direct` field records
+  the open's real outcome on every platform, so a refused or downgraded run reports `0`.
 - **No iOS target.** The core is portable C++ and the streaming path has no Android dependency, but
   there is no Xcode project here and iOS does not run command-line binaries, so there is no
   supported way to run or benchmark the engine on an iPhone or iPad.

docs/telemetry.md

@@ -291,6 +291,11 @@ effective top-k after any override. Fields to read carefully:
   not reproducible except through `seed`.
 - `load_all=1` reads the whole expert set, so its `read_bytes` means something different from a
   selective run's.
+- `o_direct=<0|1>` is what the shard opens **achieved**, not what the flag asked for: a platform can
+  refuse the request, and the open-time verify can downgrade a shard that mis-serves it to buffered.
+  On a Mac the request is served with `F_NOCACHE` — it turns data caching off for the descriptor
+  without imposing any of Linux `O_DIRECT`'s alignment or DMA semantics — so `o_direct=1` there
+  means "uncached descriptor", never "O_DIRECT".
 - `mtp=1` means the run used the model's MTP head to draft and verified a whole group per decode.
   No weight is skipped and nothing is approximated, but the text is **not** guaranteed identical to
   an unspeculated greedy run: a verify decode is a wide batch, and batch width moves the last bits on