97 commits

This branch

This branch

All branches

Author	SHA1	Message	Date
TangJingqi	170b7a6001	fix server don't accept yaml path as param; fix server static cache device problem	2024-08-21 14:19:43 +08:00
Azure	4358722891	Merge pull request #41 from Azure-Tang/main ... Update README	2024-08-16 15:27:01 +08:00
TangJingqi	4a0f1cbbfa	Update readme	2024-08-16 15:22:16 +08:00
Azure	f681eb4a0f	Merge branch 'kvcache-ai:main' into main	2024-08-16 15:21:41 +08:00
Azure	b25ad4ec32	Merge pull request #39 from Azure-Tang/develop-0.1.2 ... [fix] fix broken link in tutorial.	2024-08-16 11:29:11 +08:00
TangJingqi	4f87756c2e	fix broken link	2024-08-16 11:10:30 +08:00
TangJingqi	7199699d78	Fix the broken link	2024-08-16 11:00:06 +08:00
Azure	e81fc482ff	update README	2024-08-15 16:59:26 +00:00
Azure	6bcec86fa0	update README	2024-08-15 16:54:31 +00:00
UnicornChan	77a34c289c	Merge pull request #36 from kvcache-ai/develop-0.1.2 ... Release v0.1.2	2024-08-15 20:59:50 +08:00
UnicornChan	395cd3e786	Merge pull request #35 from Azure-Tang/develop-0.1.2 ... [update] Update readme; Add tutorial	2024-08-15 20:56:22 +08:00
TangJingqi	de3faaf55d	Update readme; add pipeline tutorial; add detailed inject tutorial	2024-08-15 20:42:54 +08:00
UnicornChan	c9bf79299b	Merge pull request #34 from Azure-Tang/develop-0.1.2 ... [fix] format classes and files name	2024-08-15 11:28:19 +08:00
TangJingqi	c47205dce9	fix name	2024-08-15 11:25:12 +08:00
TangJingqi	67043b4b5c	[fix] format classes and files name	2024-08-15 10:44:59 +08:00
chenxl	1db4a67dca	[feature] add github action for pre compile	2024-08-14 16:54:50 +00:00
Atream	412055d450	[feature] experts can be injected using CPUInfer ... [fix] fix ktransformers interface when use new CUDAGraphRunner [fix] fix YAML and optimize logic, the top rule has the highest priority	2024-08-14 16:10:54 +08:00
UnicornChan	80815dbc50	Merge pull request #30 from BITcyman/feature-lc-dequantize_q2k_q3k_gpu ... [feature] support q2_k & q3_k dequantize on gpu	2024-08-12 21:02:32 +08:00
BITcyman	7c4cb520bd	[feature] support q2_k & q3_k dequantize on gpu	2024-08-12 12:53:12 +00:00
chenxl	650c368c18	Merge remote-tracking branch 'upstream/main' into develop-0.1.2	2024-08-12 12:31:49 +00:00
UnicornChan	44f57270c9	Merge pull request #29 from kvcache-ai/fix]-fix-linux-mutex ... Update task_queue.h	2024-08-12 20:14:04 +08:00
Atream	3c675af61a	Update task_queue.h	2024-08-12 20:06:19 +08:00
chenxl	f5f79f5c0e	[ADD] support multi-gpu qlen>1 q5_k	2024-08-12 11:41:26 +00:00
UnicornChan	f293803156	Merge pull request #27 from chenht2022/develop-0.1.2 ... [Feature] towards 0.1.2	2024-08-09 17:57:27 +08:00
UnicornChan	cb7f8e7817	Merge pull request #26 from kvcache-ai/windows ... [fix] linux and windows can all find CPUInfer in current Directory	2024-08-09 17:57:00 +08:00
chenxl	782a17e4e6	[feature] add bat for windows, update readme	2024-08-09 09:39:42 +00:00
chenht2022	c1cc7d2cd2	1) Linear and MLP operators support qlen>1; 2) All operators now share a single memory buffer; 3) Refactor CPUInfer submit/sync logic.	2024-08-08 09:04:36 +00:00
Atream	1f92f7cc61	[fix] linux and windows can all find CPUInfer in current Directory	2024-08-08 16:02:42 +08:00
UnicornChan	11544ef2b0	Merge pull request #25 from kvcache-ai/windows ... Windows Support	2024-08-08 16:01:54 +08:00
Atream	0e613b602d	[fix] recover fp16 support	2024-08-08 15:42:21 +08:00
chenxl	1d9d397525	fix some bug in compile in linux	2024-08-08 15:34:19 +08:00
Atream	0a2fd52cea	support windows support q4_0 and q5_0 dequant on cpu Add CopyRight from pygguf(It was added before, but disappear after merge). Add some TODO in the code.	2024-08-08 15:34:02 +08:00
UnicornChan	442e13bc97	Merge pull request #18 from UnicornChan/update-docker-readme ... update docker.md to support docker pull image	2024-08-01 15:40:47 +08:00
chenxl	69fa2f4395	update docker.md support docker pull image	2024-08-01 07:39:48 +00:00
UnicornChan	5e83bc0c82	Merge pull request #17 from UnicornChan/feature-support-multi-instruct ... Feature support multi instruct	2024-08-01 12:29:07 +08:00
chenxl	86ba1336a9	[feature] add support for building docker image	2024-08-01 04:01:00 +00:00
chenxl	112cb3c962	[feature] support python 310 and multi instruction	2024-07-31 13:58:17 +00:00
UnicornChan	25620829ce	Merge pull request #14 from UnicornChan/feature-support-pypi ... [feature] support for pypi install	2024-07-29 19:58:18 +08:00
chenxl	dd18a11cab	[feature] support for pypi install	2024-07-29 11:51:28 +00:00
Allen	a25320b703	Update README.md ... update bigcodebench result picture	2024-07-27 23:40:08 +08:00
UnicornChan	8629301b7f	Update README.md ... fix_can't access picture	2024-07-27 23:36:13 +08:00
UnicornChan	de13c3e7cd	Update README.md ... fix can't access picture	2024-07-27 23:27:32 +08:00
UnicornChan	1ac7bfd75e	Update README.md ... fix video can't access	2024-07-27 23:19:49 +08:00
Mingxing Zhang	6977ebb555	Merge pull request #7 from kvcache-ai/james0zan-patch-1 ... Update video	2024-07-27 23:12:59 +08:00
Mingxing Zhang	935c28c277	Update README.md	2024-07-27 23:12:00 +08:00
Mingxing Zhang	01df4a6b3f	Update README.md ... update videos	2024-07-27 23:07:31 +08:00
chenxl	18c42e67df	Initial commit	2024-07-27 16:06:58 +08:00