vrr/prima.cpp
2
0
Fork 0
mirror of https://github.com/Lizonghang/prima.cpp.git synced 2025-09-06 01:09:02 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
4175 commits 6 branches 0 tags 73 MiB
Commit graph

4175 commits

This branch
This branch
All branches
Author SHA1 Message Date
Lizonghang
d8aea899d1 fix n_seq_id and seq_id 2025-06-06 23:58:03 +04:00
Lizonghang
a1a2238831 add batch_all.n_seq_id and batch_all.seq_id to sync_meta 2025-06-06 23:36:53 +04:00
Lizonghang
68ecc8509d add batch_all.logits to sync_meta 2025-06-06 22:58:48 +04:00
Lizonghang
500e066a2f fix batch decoding and dynamic batching 2025-06-06 16:53:22 +04:00
Lizonghang
ef1e10101e add test for IQ1 and doc for device selection 2025-06-04 15:12:00 +04:00
Lizonghang
27756ee182 fix: enable rolling back set assignment when all devices are assigned to M4 but no feasible solutions 2025-06-04 15:11:29 +04:00
Li, Zonghang
6439090920 reformat code 2025-06-03 23:53:24 +04:00
Li, Zonghang
b6fdbd541b Merge branch 'dev' of github.com:Lizonghang/prima.cpp into dev 2025-06-03 18:20:17 +04:00
Lizonghang
9f0ec78a4b Merge branch 'dev' of github.com:Lizonghang/prima.cpp into dev 2025-06-03 18:18:53 +04:00
Li, Zonghang
a01fafd126 Merge branch 'main' into dev 2025-06-03 17:56:47 +04:00
Li, Zonghang
1b3b6a506f fix: add warm-up in profiling to prevent init delay 2025-06-03 17:10:09 +04:00
Li, Zonghang
b30f749e5e fix n_embd cannot be divided by quantized block size 2025-06-03 14:06:31 +04:00
Li, Zonghang
e25c739ecf
Merge pull request #19 from yezhizi/feat/auto-exit 
feat: ranks with only 1 layer auto-exit and rebuild topology
 2025-05-20 02:04:50 +08:00
Li, Zonghang
7b0ededd24
Merge branch 'dev' into feat/auto-exit 2025-05-20 02:04:14 +08:00
Lizonghang
421b3deca5 fix llama-cli pos sync 2025-05-19 18:08:27 +04:00
Lizonghang
c54a6a0132 fix context shifting 2025-05-19 16:58:35 +04:00
DeEMO
34eaa8224d fix: handle socket closure and connection in llama_rebuild_topo 
Signed-off-by: DeEMO <yzzxrx@gmail.com>
 2025-05-19 09:22:35 +00:00
DeEMO
8b61cb2fa4 fix: adapt the new topo 
Signed-off-by: DeEMO <yzzxrx@gmail.com>
 2025-05-19 09:22:29 +00:00
DeEMO
df16b1876f refactor: add zmq helper to generate message 
Signed-off-by: DeEMO <yzzxrx@gmail.com>
 2025-05-19 09:22:24 +00:00
DeEMO
0ad009a2f4 fix: update serialization and deserialization for next_ip in device_info 
Signed-off-by: DeEMO <yzzxrx@gmail.com>
 2025-05-19 09:22:16 +00:00
DeEMO
4b36aef157 fix some bugs 
Signed-off-by: DeEMO <yzzxrx@gmail.com>
 2025-05-19 09:22:08 +00:00
DeEMO
cc46aa9828 update rank and n_world 
Signed-off-by: DeEMO <yzzxrx@gmail.com>
 2025-05-19 09:22:02 +00:00
DeEMO
fdd6694633 add topo rebuild 
Signed-off-by: DeEMO <yzzxrx@gmail.com>
 2025-05-19 09:21:53 +00:00
DeEMO
26bb86c09b Add tune_layer_allocation 
Signed-off-by: DeEMO <yzzxrx@gmail.com>
 2025-05-19 09:21:22 +00:00
Lizonghang
07c4966a80 reduce fio data size to 1gb to speed up profiling 2025-05-14 21:26:01 +04:00
Lizonghang
2cc01483fd support server mode 2025-05-14 18:28:46 +04:00
Lizonghang
ebd09fc83c Merge branch 'dev' 2025-05-14 14:19:53 +04:00
Lizonghang
258fb2d06b add QA: How to manually profile a device 2025-05-14 14:19:20 +04:00
Lizonghang
2fbc0c8da3 fix: reset -ngl to 0 when GPU is not used and reformat code 2025-05-14 13:27:20 +04:00
Li, Zonghang
b44187e3af
Merge pull request #9 from yezhizi/rm-profile 
Remove unnecessary profiling when `-lw` is specified
 2025-05-14 10:05:49 +08:00
Li, Zonghang
e2de4511c5
Update README.md 2025-05-11 18:15:39 +08:00
Li, Zonghang
ca95b1efad
Merge pull request #11 from eltociear/patch-1 
chore: update debug-test.sh
 2025-04-22 00:32:19 +08:00
Ikko Eltociear Ashimine
fa1444ab3b
chore: update debug-test.sh 
compliation -> compilation
 2025-04-21 23:40:40 +09:00
DeEMO
cf4fa04980 Add an independent profile tool 2025-04-18 03:27:20 +00:00
DeEMO
168c14f4e8 remove unnecessary profile when --lw is specified 2025-04-17 13:49:09 +00:00
DeEMO
9cb87f7923 add fio file to gitignore 2025-04-17 13:44:23 +00:00
leeetao
fc1e2d3fc6 Added support for iq1s and iq1m quantization type 2025-04-17 10:27:53 +00:00
leeetao
b212d74dc3 update Readme.md 2025-04-17 09:17:11 +00:00
Zonghang Li
f9702ec4c0 update README.md 2025-04-16 15:55:43 +04:00
Zonghang Li
5a5f103833 fix q6k and q80 2025-04-16 08:55:07 +04:00
Li, Zonghang
b59d6d9cdf
Update README.md 2025-04-15 09:59:08 +08:00
Li, Zonghang
6d13836c44
Update README.md 2025-04-11 01:41:43 +08:00
Li, Zonghang
4845abf25e
Update README.md 2025-04-11 01:20:36 +08:00
Li, Zonghang
4ab9087d69
Update LICENSE 2025-04-09 21:42:55 +08:00
Lizonghang
e48b804730 update README.md 2025-04-09 13:55:30 +04:00
Li, Zonghang
55f8dc588f
Update README.md 2025-04-09 10:56:25 +08:00
Lizonghang
e421d788d3 update README 2025-04-08 23:15:43 +04:00
Lizonghang
03ff9a7654 update README 2025-04-07 23:28:01 +04:00
Li, Zonghang
a3a1f4499b
Update README.md 2025-04-07 22:14:44 +08:00
Li, Zonghang
98d73778a6
Update README.md 2025-04-07 22:13:31 +08:00