7 commits

Author	SHA1	Message	Date
Atream	5ec33d046d	optimize gguf dequant, save mem, support Q2_K ... use marlin for lm_head, lm_head only calc last token for prefill extend context window to 19K for DeepSeek-V3/R1 within 24GB VRAM	2025-02-22 06:13:01 +00:00
liam	098602b08f	⚡ v0.2 ongoing	2025-02-09 22:41:14 +08:00
liam	3dca28d23b	⚡ fix moe.cpp int overflow problem	2025-02-06 22:39:16 +08:00
chenxl	4d1d561d28	[feature] release 0.1.3	2024-08-28 16:11:43 +00:00
chenxl	f5f79f5c0e	[ADD] support multi-gpu qlen>1 q5_k	2024-08-12 11:41:26 +00:00
chenht2022	c1cc7d2cd2	1) Linear and MLP operators support qlen>1; 2) All operators now share a single memory buffer; 3) Refactor CPUInfer submit/sync logic.	2024-08-08 09:04:36 +00:00
chenxl	18c42e67df	Initial commit	2024-07-27 16:06:58 +08:00