vrr/kvcache-ai-ktransformers
2
0
Fork 0
mirror of https://github.com/kvcache-ai/ktransformers.git synced 2025-09-16 18:19:42 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 2

support smt and glm4

Browse source
This commit is contained in:
djw 2025-07-24 12:31:01 +00:00
parent 613f0b7c37
commit 590fcb41cd
5 changed files with 95 additions and 7 deletions
Download patch file Download diff file Expand all files Collapse all files

2
ktransformers/optimize/optimize_rules/Glm4Moe-serve.yaml
View file

@ -49,7 +49,7 @@
- match:
name: "^model\\.layers\\..*\\.mlp\\.experts$"
replace:
class: ktransformers.operators.experts.KTransformersExpertsV2 # custom MoE Kernel with expert paralleism
class: ktransformers.operators.experts.KGlm4Experts # custom MoE Kernel with expert paralleism
kwargs:
prefill_device: "cuda"
prefill_op: None