9 commits

Author	SHA1	Message	Date
UnicornChan	a81a7ffe21	Merge pull request #77 from TKONIY/fix-prefill-and-generate ... Fix: Wrong type of token list returned by prefill_and_generate	2024-10-09 19:04:27 +08:00
yangshen	ee72cee050	Fix: the tokens return by prefill_and_generate	2024-09-05 05:29:23 +00:00
Yap Sok Ann	be356c1b8d	Support IQ4_XS dequantize	2024-09-02 09:10:19 +07:00
chenxl	4d1d561d28	[feature] release 0.1.3	2024-08-28 16:11:43 +00:00
molamooo	29f4151ebc	[fix] f16 dequantize device ignored	2024-08-22 15:10:06 +08:00
Atream	412055d450	[feature] experts can be injected using CPUInfer ... [fix] fix ktransformers interface when use new CUDAGraphRunner [fix] fix YAML and optimize logic, the top rule has the highest priority	2024-08-14 16:10:54 +08:00
BITcyman	7c4cb520bd	[feature] support q2_k & q3_k dequantize on gpu	2024-08-12 12:53:12 +00:00
chenxl	f5f79f5c0e	[ADD] support multi-gpu qlen>1 q5_k	2024-08-12 11:41:26 +00:00
chenxl	18c42e67df	Initial commit	2024-07-27 16:06:58 +08:00