mirror of
https://github.com/kvcache-ai/ktransformers.git
synced 2026-04-28 11:49:51 +00:00
fcf8882075
Some checks are pending
Book-CI / test-2 (push) Waiting to run
Book-CI / test (push) Waiting to run
Book-CI / test-1 (push) Waiting to run
Deploy / deploy (macos-latest) (push) Waiting to run
Deploy / deploy (ubuntu-latest) (push) Waiting to run
Deploy / deploy (windows-latest) (push) Waiting to run
* support Kimi-K2-Thinking original weight fix amx kernel bug * update k2 avx kernel. * feat: add CPUInfer write buffer task * [feat]: add kimi k2 cpu write buffer support - Implement write_weights_to_buffer function in k2-moe.hpp for extracting GPU expert weights - Fix down (w2) weight column-wise slicing for different TP configurations - Support three TP scenarios: cpu_tp == gpu_tp, cpu_tp > gpu_tp, cpu_tp < gpu_tp - Add comprehensive test cases for weight extraction validation - Ensure compatibility with Kimi model's MoE architecture * [fix]: correct write_weight_scale_to_buffer expert offset calculation Fixed the bug in write_weight_scale_to_buffer_task where expert offsets in GPU buffers were incorrectly calculated. Changed from using per_expert_gpu sizes to using full gpu_tp sizes, ensuring correct memory layout for multi-expert scenarios. Also added benchmark scripts for k2 moe and write buffer operations, and cleaned up debug output in test files. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * [feat]: add write buffer wrapper * [fix] fix comment --------- Co-authored-by: ouqingliang <1692110604@qq.com> Co-authored-by: Claude <noreply@anthropic.com> |
||
|---|---|---|
| .. | ||
| .gitignore | ||
| configuration_deepseek_v3.py | ||
| modeling_deepseek_v3.py | ||
| repro_llamafile_re.py | ||
| test-debug.py | ||
| test_apply_rope.py | ||
| test_attention.py | ||
| test_awq_moe_amx.py | ||
| test_deepseekv3.py | ||
| test_deepseekv3_prefill.py | ||
| test_deepseekv3_prefill_speed.py | ||
| test_gate.py | ||
| test_k2_moe_amx.py | ||
| test_k2_write_buffer.py | ||
| test_linear.py | ||
| test_mla.py | ||
| test_mla_qlen.py | ||
| test_mla_quant.py | ||
| test_mla_simple.py | ||
| test_mla_torch.py | ||
| test_mlp.py | ||
| test_moe.py | ||
| test_moe_amx.py | ||
| test_moe_kernel.py | ||
| test_moe_kml.py | ||
| test_rope.cpp | ||
| test_rope.py | ||
| test_softmax.py | ||
| torch_attention.py | ||