26 commits

Author	SHA1	Message	Date
DDong Jianwei	95d937c51d	tmp	2025-02-23 18:51:42 +08:00
Atream	5ec33d046d	optimize gguf dequant, save mem, support Q2_K ... use marlin for lm_head, lm_head only calc last token for prefill extend context window to 19K for DeepSeek-V3/R1 within 24GB VRAM	2025-02-22 06:13:01 +00:00
Atream	7e1fe256c8	optimize GPU	2025-02-21 05:06:57 +00:00
Atream	c189d55bd1	toy support for experts on GPU, no CUDA Graph	2025-02-15 15:16:00 +00:00
Azure	b7653b9c4f	add V3/R1 8 gpu yaml example	2025-02-14 02:56:13 +00:00
MorphisZhang	aea4243712	Add optimization config for Deepseek V3/R1 with 4 GPUs	2025-02-13 16:32:28 +08:00
Azure	0564ac8465	update marlin expert example	2025-02-12 04:11:00 +00:00
liam	83401dbb3b	⚡ ready to publish	2025-02-10 12:29:23 +08:00
Azure	c4d9bc6670	support KExpertsMarlin backend	2025-02-07 05:57:40 +00:00
Azure	ee24a27001	update v3 single gpu rule yaml;	2025-02-04 16:14:35 +00:00
Azure	907251c743	done support deepseekv3	2025-02-04 15:53:38 +00:00
Azure	f748cd29f0	fix rope; update moegate	2025-02-01 18:05:45 +00:00
Azure	f873558a89	update rope calculation; update modeling.py; update gate for moe	2025-02-01 07:32:21 +00:00
Azure	476b1d8dc6	support deepseekv3; runable but have precition problem	2025-01-31 08:27:24 +00:00
anyanqilin	2d67016d14	wjh-change	2024-11-04 14:02:19 +08:00
xhedit	234faf7987	typo fix: KMisrtal -> KMistral	2024-09-12 15:58:01 +00:00
chenxl	49cce0c437	[fix] bugs about Qwen57B, install requirement, Dockerfile	2024-08-30 09:51:32 +00:00
TangJingqi	8747c099f2	update yaml example; update version idx; update docker file	2024-08-29 22:39:20 +08:00
TangJingqi	abd4214b56	fix readme; adjust param	2024-08-29 10:40:08 +08:00
chenxl	4d1d561d28	[feature] release 0.1.3	2024-08-28 16:11:43 +00:00
TangJingqi	de3faaf55d	Update readme; add pipeline tutorial; add detailed inject tutorial	2024-08-15 20:42:54 +08:00
TangJingqi	c47205dce9	fix name	2024-08-15 11:25:12 +08:00
TangJingqi	67043b4b5c	[fix] format classes and files name	2024-08-15 10:44:59 +08:00
Atream	412055d450	[feature] experts can be injected using CPUInfer ... [fix] fix ktransformers interface when use new CUDAGraphRunner [fix] fix YAML and optimize logic, the top rule has the highest priority	2024-08-14 16:10:54 +08:00
chenxl	f5f79f5c0e	[ADD] support multi-gpu qlen>1 q5_k	2024-08-12 11:41:26 +00:00
chenxl	18c42e67df	Initial commit	2024-07-27 16:06:58 +08:00