vrr/kvcache-ai-ktransformers
2
0
Fork 0
mirror of https://github.com/kvcache-ai/ktransformers.git synced 2026-04-28 11:49:51 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 4
kvcache-ai-ktransformers/kt-kernel/python/utils
History
ErvinXie 3903c9afcc
(kt-kernel): add numa_nodes parameter for explicit NUMA node mapping (#1891) 
Add numa_nodes parameter to BaseMoEWrapper and all subclasses, allowing
users to explicitly specify which NUMA node IDs to use for subpool
mapping instead of always defaulting to sequential [0, 1, ..., N-1].

This enables running multiple KTransformers instances on different NUMA
nodes of the same machine, e.g. --kt-threadpool-count 1 --kt-numa-nodes 1
to bind to NUMA node 1. Previously this required external numactl
workarounds since subpool_numa_map was hardcoded to start from 0.
2026-03-31 10:27:50 +08:00
..
__init__.py Kt minimax (#1742) 2025-12-24 15:39:44 +08:00
amx.py (kt-kernel): add numa_nodes parameter for explicit NUMA node mapping (#1891) 2026-03-31 10:27:50 +08:00
llamafile.py (kt-kernel): add numa_nodes parameter for explicit NUMA node mapping (#1891) 2026-03-31 10:27:50 +08:00
loader.py [feat](kt-kernel): support avx2 only inference for bf16 fp8 and gptq int4 (#1892) 2026-03-27 14:45:02 +08:00
moe_kernel.py (kt-kernel): add numa_nodes parameter for explicit NUMA node mapping (#1891) 2026-03-31 10:27:50 +08:00