vrr/kvcache-ai-ktransformers
2
0
Fork 0
mirror of https://github.com/kvcache-ai/ktransformers.git synced 2025-09-10 06:14:58 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

support npu

Browse source
This commit is contained in:
djw 2025-07-21 12:26:14 +00:00
parent dd0e41b3b8
commit 7d51a13c9b
34 changed files with 14004 additions and 5626 deletions
Download patch file Download diff file Expand all files Collapse all files

5
ktransformers/operators/flashinfer_batch_prefill_wrapper.py
View file

@ -1,5 +1,8 @@
import torch
import flashinfer
try:
import flashinfer
except:
Warning("flashinfer not found, if you are using npu, ignore this.")
import gc
try:
from flash_attn import flash_attn_with_kvcache