1253 commits

This branch

This branch

All branches

Author	SHA1	Message	Date
ZiWei Yuan	be6db6f46b	[docs]: improve structure for kt-kernel (#1624) ... * [docs]: improve structure for kt-kernel * Update doc/en/kt-kernel/README.md	2025-11-16 13:21:41 +08:00
ZiWei Yuan	133eea037c	[docs]: improve docs structure (#1623)	2025-11-16 12:40:59 +08:00
ZiWei Yuan	c2d2edbeef	[docs]: update the web docs structure (#1622)	2025-11-16 12:09:44 +08:00
ZiWei Yuan	f914bc4581	[docs]: add issue config (#1621) ... * [docs]: add issue config * [docs]: consider to force the issue template	2025-11-16 11:50:43 +08:00
ZiWei Yuan	ac0e449980	[docs]: add code of conduct & pr template (#1619)	2025-11-15 22:45:53 +08:00
ZiWei Yuan	e0e2429748	[ci]: update issue template & security & license (#1617)	2025-11-15 22:29:01 +08:00
ZiWei Yuan	550e4986f5	[docs]: update README.md (#1616) ... * [docs]: update README.md	2025-11-15 20:56:26 +08:00
ZiWei Yuan	7c2ad6dbca	[docs]: update README.md (#1614) ... * [docs]: update README.md	2025-11-15 18:34:27 +08:00
ZiWei Yuan	aef6672dd8	[docs]: add contribuing guide and add hooks install (#1613) ... * [feat]: update kt-kernel hooks and add contribution guide * [docs]: add contributing guide * [style]: format the python file and cpp file in kt-kernel	2025-11-15 18:26:49 +08:00
ZiWei Yuan	c32fefb1cd	[doc]: update web doc and kt-kernel doc (#1609) ... * [doc]: update web doc and kt-kernel doc * [doc](book.toml): add book.toml for rust book compile	2025-11-13 20:44:13 +08:00
Jiaqi Liao	4bd0fe812b	docs(kt-kernel): improve SGLang integration documentation and fix syntax errors (#1607) ... - Clarified instructions for SGLang integration with kt-kernel	2025-11-13 19:23:00 +08:00
Jiaqi Liao	13b8ddecd9	AMXMoEWrapper -> KTMoEWrapper (#1604) ... fix import KTMoEWrapper	2025-11-12 16:34:54 +08:00
Jiaqi Liao	02801c3c4e	fix kt-kernel installation issue (#1603) ... * update README for kt-kernel for installation issues * update install.sh * update install.sh * update install.sh * update install.sh * update install.sh * update README * fix sudo issue * fix install issue * fix import issue * fix import issue * update install.sh * fix import issue * fix import issue * fix import issue * update README * update install * update install	2025-11-12 15:56:02 +08:00
Peilin Li	26b86aa96b	Add warning for CUDA installation step in README (#1599) ... Added a warning about skipping the CUDA installation step.	2025-11-12 11:14:50 +08:00
Peilin Li	148a030026	upload hands-on tutorial with KTransformers-FT, especially in customize your KT-FT+LLaMA-Factory (#1597) ... * Add files via upload * upload hands-on tutorial for KTransformers-FT	2025-11-11 20:54:41 +08:00
Jiaqi Liao	d483147307	Fix kt-kernel compile issue (#1595) ... * update install.sh * fix import issue * update README	2025-11-11 19:30:27 +08:00
ZiWei Yuan	a6bb7651f8	[build]: amd for llamafile right now (#1594)	2025-11-11 18:08:03 +08:00
Jiaqi Liao	bb1a667169	fix hwloc README (#1592)	2025-11-11 15:28:57 +08:00
Jiaqi Liao	743f6f719b	update README for kt-kernel for installation issues (#1590)	2025-11-11 14:53:45 +08:00
Jiaqi Liao	94c25626dc	Fix kt-kernel for new wrapper (#1588) ... * update README for kt-kernel * style: format C++ and Python code in kt-kernel - Format C++ files: task_queue, ext_bindings, and MoE operators - Format Python utility modules: amx, llamafile, and loader - Improve code readability and consistency	2025-11-10 21:47:34 +08:00
Jiaqi Liao	9bc00e587b	Refactor KTMoEWrapper backend (#1587) ... * universal backend for cpu inference * expert defer	2025-11-10 20:26:15 +08:00
Jiaqi Liao	956d19d2d8	Fix git submodule (#1586) ... * refactor repo * fix README * fix gitsubmodule	2025-11-10 19:10:13 +08:00
ErvinXie	5179f0d634	Add roadmap link to README (#1585)	2025-11-10 18:15:53 +08:00
Jiaqi Liao	07322ca2bd	Refactor: restructure repository to focus on kt-kernel and KT-SFT modules (#1583) ... * refactor repo * fix README	2025-11-10 17:57:48 +08:00
ErvinXie	2cb1674020	Fix image reference in README.md (#1584) ... Updated image reference in README for heterogeneous computing.	2025-11-10 17:53:41 +08:00
Jiaqi Liao	57d14d22bc	Refactor: restructure repository to focus on kt-kernel and KT-SFT modulesq recon (#1581) ... * refactor: move legacy code to archive/ directory - Moved ktransformers, csrc, third_party, merge_tensors to archive/ - Moved build scripts and configurations to archive/ - Kept kt-kernel, KT-SFT, doc, and README files in root - Preserved complete git history for all moved files * refactor: restructure repository to focus on kt-kernel and KT-SFT modules * fix README * fix README * fix README * fix README * docs: add performance benchmarks to kt-kernel section Add comprehensive performance data for kt-kernel to match KT-SFT's presentation: - AMX kernel optimization: 21.3 TFLOPS (3.9× faster than PyTorch) - Prefill phase: up to 20× speedup vs baseline - Decode phase: up to 4× speedup - NUMA optimization: up to 63% throughput improvement - Multi-GPU (8×L20): 227.85 tokens/s total throughput with DeepSeek-R1 FP8 Source: https://lmsys.org/blog/2025-10-22-KTransformers/ This provides users with concrete performance metrics for both core modules, making it easier to understand the capabilities of each component. * refactor: improve kt-kernel performance data with specific hardware and models Replace generic performance descriptions with concrete benchmarks: - Specify exact hardware: 8×L20 GPU + Xeon Gold 6454S, Single/Dual-socket Xeon + AMX - Include specific models: DeepSeek-R1-0528 (FP8), DeepSeek-V3 (671B) - Show detailed metrics: total throughput, output throughput, concurrency details - Match KT-SFT presentation style for consistency This provides users with actionable performance data they can use to evaluate hardware requirements and expected performance for their use cases. * fix README * docs: clean up performance table and improve formatting * add pic for README * refactor: simplify .gitmodules and backup legacy submodules - Remove 7 legacy submodules from root .gitmodules (archive/third_party/*) - Keep only 2 active submodules for kt-kernel (llama.cpp, pybind11) - Backup complete .gitmodules to archive/.gitmodules - Add documentation in archive/README.md for researchers who need legacy submodules This reduces initial clone size by ~500MB and avoids downloading unused dependencies. * refactor: move doc/ back to root directory Keep documentation in root for easier access and maintenance. * refactor: consolidate all images to doc/assets/ - Move kt-kernel/assets/heterogeneous_computing.png to doc/assets/ - Remove KT-SFT/assets/ (images already in doc/assets/) - Update KT-SFT/README.md image references to ../doc/assets/ - Eliminates ~7.9MB image duplication - Centralizes all documentation assets in one location * fix pic path for README	2025-11-10 17:42:26 +08:00
yuxuan-z	8729435d85	fix: add missing CMakeLists.txt for KT-SFT (#1576)	2025-11-10 16:51:24 +08:00
Wenzhang-Chen	62b7b28a16	fix typo (#1452)	2025-11-10 16:08:04 +08:00
Atream	a9276db254	Merge pull request #1574 from kvcache-ai/Atream-patch-12 ... Change attention backend to 'flashinfer' in launch command	2025-11-08 20:57:03 +08:00
Atream	b67cc4095d	Change attention backend to 'flashinfer' in launch command ... Updated the launch command to include 'flashinfer' as the attention backend.	2025-11-08 20:56:09 +08:00
Peilin Li	0dd4287ff0	Merge pull request #1573 from kvcache-ai/JimmyPeilinLi-patch-4 ... add deepseek-v2 KT-SFT yaml	2025-11-08 17:07:51 +08:00
Peilin Li	388e3eb0c1	add deepseek-v2 KT-SFT yaml	2025-11-08 17:06:13 +08:00
Peilin Li	f4fe137023	Merge pull request #1572 from JimmyPeilinLi/main ... fix: remove py310 as guide	2025-11-08 16:57:10 +08:00
Atream	c56265e541	Merge pull request #1571 from kvcache-ai/Atream-patch-11 ... Simplify launch command by removing unused option	2025-11-08 16:54:49 +08:00
JimmyPeilinLi	1c08a4f0fb	fix: remove py310 as guide	2025-11-08 08:54:32 +00:00
Atream	0651dbda04	Simplify launch command by removing unused option ... Removed the unused '--attention-backend triton' option from the launch command.	2025-11-08 16:54:18 +08:00
Jianwei Dong	8c99148c9c	Merge pull request #1569 from KMSorSMS/main ... [fix]: fix precision of amx int8	2025-11-08 13:38:58 +08:00
KMSorSMS	62d8685699	[fix]: update moe's physical to logical map	2025-11-07 16:29:04 +00:00
KMSorSMS	2641d15383	[fix]: add amx optimization	2025-11-07 15:51:56 +00:00
KMSorSMS	9509220b64	[feat]: Merge branch 'main' of https://github.com/kvcache-ai/ktransformers	2025-11-07 14:56:43 +00:00
KMSorSMS	7b88bb3d39	[fix]: fix precision	2025-11-07 14:56:05 +00:00
Oql	34c71ba8bf	Merge pull request #1568 from kvcache-ai/add_bf16_scripts ... add convert_moe_to_bf16.py	2025-11-07 17:55:38 +08:00
ouqingliang	a18f007d45	add convert_moe_to_bf16.py	2025-11-07 09:53:19 +00:00
Atream	32fab532c6	Merge pull request #1564 from kvcache-ai/Atream-patch-10 ... Fix download link for Kimi-K2-Thinking weights	2025-11-06 19:07:38 +08:00
Atream	d6ee384fe2	Fix download link for Kimi-K2-Thinking weights ... Updated the download link for AMX INT4 quantized weights.	2025-11-06 19:07:15 +08:00
Atream	f3c4dbe181	Merge pull request #1562 from kvcache-ai/kimi-k2-thinking ... Kimi k2 thinking	2025-11-06 18:17:46 +08:00
Atream	86229c852d	Add update for Kimi-K2-Thinking support	2025-11-06 17:56:46 +08:00
Atream	d419024bb4	Add KTransformers SGLang inference documentation ... Add documentation for KTransformers SGLang inference deployment, including installation steps, model download links, server launch instructions, and performance benchmarks.	2025-11-06 17:53:58 +08:00
Peilin Li	1ab570b5ca	Merge pull request #1561 from kvcache-ai/JimmyPeilinLi-patch-3 ... Update SFT Installation Guide for KimiK2	2025-11-06 17:34:33 +08:00
Peilin Li	803e645bc1	Update SFT Installation Guide for KimiK2 ... Added installation instructions and usage examples for KimiK2.	2025-11-06 17:34:21 +08:00