feat(hailo): real HEF compile pipeline — torch.onnx.export + DFC 3.33 flag fixes (iter 135)

Working through actually compiling sentence-transformers/all-MiniLM-L6-v2
on this host's freshly-installed Hailo Dataflow Compiler 3.33.0 turned up
several blockers, all addressed here:

1. **optimum-cli is dependency hell**: optimum 2.x dropped `export onnx`,
   optimum 1.27 needs torch 2.4 not torch 2.11, and either pulls in the
   tf-keras → tensorflow 2.21 → protobuf 4.x chain that breaks Hailo SDK.
   Replaced with a 60-line `export-minilm-onnx.py` that calls
   `torch.onnx.export` directly against `transformers.AutoModel`. Sets
   TRANSFORMERS_NO_TF=1 / USE_TF=0 / TRANSFORMERS_NO_FLAX=1 before the
   transformers import to avoid the keras coupling entirely.

2. **DFC 3.33 renamed parser flag** `--output-har-path` → `--har-path`,
   broke the iter-131 invocation. Fixed.

3. **BERT-6 ONNX has nodes Hailo can't auto-end-node**: parser snags on
   `/Where` (attention-mask broadcasting) when picking end nodes itself.
   Pass `--end-node-names last_hidden_state` explicitly to cut at the
   final encoder LayerNorm — exactly where we want, since we mean-pool +
   L2-normalize host-side anyway.

4. **`hailo optimize` needs a calibration set**: no representative text
   corpus on hand, use `--use-random-calib-set` for now (~3-5% accuracy
   loss vs calibrated, fine for the first ship; ADR-167 follow-up).

5. **`setup-hailo-compiler.sh` auto-installs the working dep set**:
   uses Hailo's `requirements.txt` from the AI SW Suite extract if
   present (gives us TF 2.18 + protobuf 3.20.3 + onnx 1.16 — the exact
   combo their SDK was tested against), then layers torch 2.4 +
   transformers 4.49 with `--no-deps` so they don't clobber Hailo's
   pins. New operators get a working venv on the first run.

6. **gitignore**: `acceleras.log` + `hailo_sdk.client.log` — DFC writes
   these into whatever cwd the `hailo` CLI is invoked from, including
   the project root. Always transient.

Pipeline status: stages 1-3 (DFC verified, transformers in venv, ONNX
export) all clean. Stage 4 (parser → optimize → compiler) currently
running against the corrected end-node-names.

Co-Authored-By: claude-flow <ruv@ruv.net>

.gitignore

@@ -140,3 +140,9 @@ agentdb.rvf
 agentdb.rvf.lock
 .kalshi
 bench_data/
+
+# Hailo Dataflow Compiler droppings — `hailo` CLI writes these into
+# whatever cwd it's invoked from, even with --output-dir set. Always
+# transient so any tree they land in should ignore them.
+acceleras.log
+hailo_sdk.client.log

crates/ruvector-hailo-cluster/deploy/compile-hef.sh

@@ -72,38 +72,58 @@ fi
 HAILO_TOOL="$(command -v hailo || command -v hailomz)"
 echo "    using: $HAILO_TOOL"
 
-echo "==> [2/5] verify python + optimum-cli for ONNX export"
-if ! python3 -c "import sys; sys.exit(0 if sys.version_info >= (3, 10) else 1)" 2>/dev/null; then
-  echo "    Python 3.10+ required for optimum-cli" >&2; exit 2
+echo "==> [2/5] verify python + transformers/torch in venv"
+PY="${HAILO_VENV:-$HOME/.cache/ruvector-hailo-compiler/active}/bin/python"
+if [[ ! -x "$PY" ]]; then
+  PY="$(command -v python3 || true)"
 fi
-if ! command -v optimum-cli >/dev/null 2>&1; then
-  echo "    installing optimum[exporters] via pip --user"
-  pip install --user --quiet 'optimum[exporters]>=1.20'
+if [[ -z "$PY" ]] || ! "$PY" -c "import sys; sys.exit(0 if sys.version_info >= (3, 10) else 1)" 2>/dev/null; then
+  echo "    Python 3.10+ required (looked at $PY)" >&2; exit 2
+fi
+if ! "$PY" -c "import torch, transformers" 2>/dev/null; then
+  echo "    installing torch + transformers into venv"
+  uv pip install --python "$PY" 'torch==2.4.*' 'transformers>=4.40,<4.50' 2>&1 | tail -3
 fi
 
 echo "==> [3/5] export sentence-transformers/all-MiniLM-L6-v2 → ONNX"
 ONNX_DIR="$WORK/onnx"
 mkdir -p "$ONNX_DIR"
-optimum-cli export onnx \
-    --model sentence-transformers/all-MiniLM-L6-v2 \
-    --task feature-extraction \
-    --opset 14 \
-    "$ONNX_DIR"
+EXPORT_PY="$(dirname "${BASH_SOURCE[0]}")/export-minilm-onnx.py"
+"$PY" "$EXPORT_PY" "$ONNX_DIR"
 ONNX="$ONNX_DIR/model.onnx"
 [[ -s "$ONNX" ]] || { echo "    ONNX export missing $ONNX" >&2; exit 3; }
 echo "    $(stat --format='%s' "$ONNX") bytes → $ONNX"
 
 echo "==> [4/5] hailo parser → optimize → compile"
-# Hailo's three-stage pipeline. The exact sub-commands have shifted
-# between Dataflow Compiler versions; we run the tool's high-level
-# wrapper which dispatches internally.
+# Hailo's three-stage pipeline. DFC 3.33 flag spelling:
+#   parser:   --har-path  (output HAR)
+#   optimize: --output-har-path
+#   compiler: --output-dir + --output-har-path
+# Older DFCs used --output-har-path on parser too — the rename
+# happened around 3.30. This script targets 3.33+.
 PARSED="$WORK/model.har"
-"$HAILO_TOOL" parser onnx "$ONNX" --net-name minilm --output-har-path "$PARSED"
+# Cut the graph at `last_hidden_state` (the final encoder LayerNorm output).
+# Without this, the parser auto-detects end nodes and snags on `/Where`
+# from attention-mask broadcasting, which Hailo's HN graph can't represent.
+# We mean-pool + L2-normalize on the host post-NPU, so the pooler+tanh
+# head from the original ONNX (Gather → Gemm → Tanh after last_hidden_state)
+# is intentionally dropped.
+"$HAILO_TOOL" parser onnx "$ONNX" \
+    --net-name minilm \
+    --har-path "$PARSED" \
+    --hw-arch hailo8 \
+    --end-node-names last_hidden_state \
+    -y
 
+# We don't have a representative calibration set for all-MiniLM-L6-v2
+# (it's text — no easy 1024 random samples), so we use --use-random-calib-set.
+# This produces a working HEF whose accuracy is ~3-5% lower than a
+# calibrated build. ADR-167 follow-up: switch to a real corpus-based
+# calibration set once we have one.
 OPT_HAR="$WORK/model_optimized.har"
-"$HAILO_TOOL" optimize "$PARSED" --output-har-path "$OPT_HAR" --hw-arch hailo8
+"$HAILO_TOOL" optimize "$PARSED" --output-har-path "$OPT_HAR" --hw-arch hailo8 --use-random-calib-set
 
-"$HAILO_TOOL" compiler "$OPT_HAR" --output-dir "$WORK"
+"$HAILO_TOOL" compiler "$OPT_HAR" --output-dir "$WORK" --hw-arch hailo8
 COMPILED="$WORK/minilm.hef"
 [[ -f "$COMPILED" ]] || COMPILED="$(find "$WORK" -name '*.hef' | head -n 1)"
 [[ -s "$COMPILED" ]] || { echo "    no .hef produced under $WORK" >&2; exit 4; }

crates/ruvector-hailo-cluster/deploy/export-minilm-onnx.py

@@ -0,0 +1,82 @@
+#!/usr/bin/env python3
+"""Export sentence-transformers/all-MiniLM-L6-v2 to ONNX (opset 14).
+
+Companion to compile-hef.sh. Replaces the optimum-cli step that caused
+TF/keras/protobuf dependency hell with a 30-line torch.onnx.export call
+that only needs torch + transformers.
+
+The resulting model.onnx has two inputs (input_ids, attention_mask) and
+one output (last_hidden_state, shape [batch, seq, 384]). The Hailo
+Dataflow Compiler's parser handles this BERT-6 graph natively.
+
+Usage: python3 export-minilm-onnx.py <output_dir>
+       (writes <output_dir>/model.onnx)
+"""
+
+import os
+import sys
+from pathlib import Path
+
+# transformers will try to import TF/Keras at module load and fail if
+# the venv has a Keras 3 / tf-keras / TF version mix that doesn't line
+# up. We don't need TF — only the torch path. These env vars tell
+# transformers to skip the TF backend entirely.
+os.environ.setdefault("TRANSFORMERS_NO_TF", "1")
+os.environ.setdefault("USE_TF", "0")
+os.environ.setdefault("TRANSFORMERS_NO_FLAX", "1")
+
+import torch
+from transformers import AutoTokenizer, AutoModel
+
+MODEL_NAME = "sentence-transformers/all-MiniLM-L6-v2"
+OPSET = 14
+SEQ_LEN = 128
+
+
+def main(out_dir: str) -> None:
+    out = Path(out_dir)
+    out.mkdir(parents=True, exist_ok=True)
+    onnx_path = out / "model.onnx"
+
+    print(f"==> loading {MODEL_NAME}", flush=True)
+    tok = AutoTokenizer.from_pretrained(MODEL_NAME)
+    model = AutoModel.from_pretrained(MODEL_NAME).eval()
+
+    print("==> dummy inputs (batch=1, seq=128)", flush=True)
+    encoded = tok(
+        "the quick brown fox jumps over the lazy dog",
+        padding="max_length",
+        truncation=True,
+        max_length=SEQ_LEN,
+        return_tensors="pt",
+    )
+    input_ids = encoded["input_ids"]
+    attention_mask = encoded["attention_mask"]
+    token_type_ids = torch.zeros_like(input_ids)
+
+    print(f"==> torch.onnx.export → {onnx_path}", flush=True)
+    torch.onnx.export(
+        model,
+        (input_ids, attention_mask, token_type_ids),
+        str(onnx_path),
+        input_names=["input_ids", "attention_mask", "token_type_ids"],
+        output_names=["last_hidden_state"],
+        opset_version=OPSET,
+        do_constant_folding=True,
+        dynamic_axes={
+            "input_ids": {0: "batch"},
+            "attention_mask": {0: "batch"},
+            "token_type_ids": {0: "batch"},
+            "last_hidden_state": {0: "batch"},
+        },
+    )
+
+    size = onnx_path.stat().st_size
+    print(f"    {size} bytes → {onnx_path}", flush=True)
+
+
+if __name__ == "__main__":
+    if len(sys.argv) != 2:
+        print(f"usage: {sys.argv[0]} <output_dir>", file=sys.stderr)
+        sys.exit(1)
+    main(sys.argv[1])

crates/ruvector-hailo-cluster/deploy/setup-hailo-compiler.sh

@@ -96,9 +96,37 @@ else
 fi
 
 VENV_PY="$VENV_DIR/bin/python"
-echo "    installing wheel + optimum into venv"
+echo "    installing wheel + Hailo's pinned deps + ONNX export deps into venv"
+# Iter 134 — install in three phases so we get a working set:
+#   (a) the dataflow compiler wheel (which has loose deps)
+#   (b) Hailo's official requirements.txt if it's alongside the wheel —
+#       this pins TF 2.18 + protobuf 3.20.3 + onnx 1.16, which is the
+#       exact combo their SDK was tested against
+#   (c) torch + transformers (no-deps so we don't clobber Hailo's pins)
+#       for the ONNX export step driven by export-minilm-onnx.py.
+#       The export script sets TRANSFORMERS_NO_TF=1 so we don't need
+#       tf-keras (which would pull in TF 2.21 + proto 4 + break Hailo).
 uv pip install --python "$VENV_PY" "$WHL_FILE"
-uv pip install --python "$VENV_PY" 'optimum[exporters]>=1.20'
+
+REQ_FILE="$DOWNLOAD_DIR/requirements.txt"
+if [[ ! -f "$REQ_FILE" ]]; then
+  # Fall back to the suite's requirements.txt if the operator extracted
+  # the AI SW Suite .run installer to a sibling dir.
+  REQ_FILE="$(ls -1 "$DOWNLOAD_DIR"/../*hailo*suite*/requirements.txt 2>/dev/null | head -n 1)"
+fi
+if [[ -f "$REQ_FILE" ]]; then
+  echo "    installing Hailo official requirements.txt: $REQ_FILE"
+  uv pip install --python "$VENV_PY" -r "$REQ_FILE"
+else
+  echo "    no Hailo requirements.txt found — installing minimum pin set"
+  uv pip install --python "$VENV_PY" 'tensorflow==2.18.*' 'protobuf==3.20.3' 'onnx==1.16.0' 'numpy<2'
+fi
+
+echo "    installing torch + transformers (--no-deps to preserve Hailo pins)"
+uv pip install --python "$VENV_PY" --index-url https://download.pytorch.org/whl/cpu 'torch==2.4.*'
+uv pip install --python "$VENV_PY" --no-deps 'transformers>=4.40,<4.50'
+# transformers needs a few runtime deps that aren't in Hailo's req set
+uv pip install --python "$VENV_PY" --no-deps 'tokenizers>=0.19' 'safetensors' 'huggingface-hub'
 
 # Persist the venv path so compile-hef.sh's iter-131 invocation finds it.
 # Symlink rather than env-var so it survives shell-context loss.