vrr/prima.cpp
2
0
Fork 0
mirror of https://github.com/Lizonghang/prima.cpp.git synced 2025-09-09 01:19:12 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
3993 commits 6 branches 0 tags 73 MiB
Commit graph

3993 commits

This branch
This branch
All branches
Author SHA1 Message Date
Lizonghang
0f73d12247 decrease compute buf from available memory 2024-11-29 11:15:54 +04:00
Lizonghang
329d084061 add illustration for memory allocation of activations 2024-11-29 10:34:21 +04:00
Lizonghang
45a1e55eec reduce kv cache from available memory 2024-11-28 20:21:21 +04:00
Lizonghang
9858d90ce4 get system readahead size automatically 2024-11-28 16:18:41 +04:00
Lizonghang
9a7bbce7ad fix t_load_us 2024-11-28 15:55:21 +04:00
Lizonghang
740f7f0b95 use multithread disk r/w test 2024-11-27 22:14:17 +04:00
Lizonghang
f7507ec20b fix disk r/w test, add disk access latency, and correct units (GB, GiB) 2024-11-27 21:36:12 +04:00
Lizonghang
9cd22177d0 remove arg test_file 2024-11-27 21:34:45 +04:00
Lizonghang
0a91ad3edc fix cuda compatibility errors 2024-11-26 22:35:58 +04:00
Zonghang Li
3f008f2ad9 add op GGML_OP_READ 2024-11-26 22:28:34 +04:00
Zonghang Li
f78c437172 add device_inp_embd_delay test, device_memory_bw test, device_cuda_memory_bw test, 2024-11-26 22:28:02 +04:00
Lizonghang
a7a95b53fe add q80xf32 and count_n_params 2024-11-24 23:11:12 +04:00
Lizonghang
3fe00a16a0 count model flops for f32xf32, f16xf32, q4kxf32, q6kxf32 2024-11-24 13:13:32 +04:00
Lizonghang
a5ba34169a add f32, f16, q4k_f32, q6k_f32 flops test and fix duplicate inp_embd in subgraphs 2024-11-23 21:36:34 +04:00
Zonghang Li
7ee1423006 add model_flops 2024-11-21 20:06:16 +04:00
Zonghang Li
80f6b72e71 remove device_flops from profiler api 2024-11-21 08:37:57 +04:00
Lizonghang
477ecf2084 add llama_model_n_flops 2024-11-20 19:40:27 +04:00
Lizonghang
10f6f92c7e add f32, f16, q8, q4k speed test for cuda 2024-11-10 23:41:13 +04:00
Lizonghang
f4260bb346 add device_flops() for cpu, metal, and cuda 2024-11-10 23:11:05 +04:00
Lizonghang
5fae6ac36f add cpu flops test 2024-11-09 20:53:42 +04:00
Lizonghang
2bd4d03aa8 add automatic layer window size assignment workflow 2024-11-08 18:21:03 +04:00
Lizonghang
53cb3a6069 synchronize device info 2024-11-07 22:02:01 +04:00
Lizonghang
ef7fdf70cc add LLAMA_API llama_profile_device 2024-11-07 09:30:39 +04:00
Zonghang Li
b922418cca convert MB to GB 2024-11-06 20:47:17 +04:00
Lizonghang
407c71ae52 add cpu and gpu profile 2024-11-06 20:42:28 +04:00
Lizonghang
4e1be1065d add memory speed test 2024-11-06 10:57:30 +04:00
Zonghang Li
9a03b52785 fix device get name on linux 2024-11-05 22:07:09 +04:00
Lizonghang
a7f3d917a1 add device get name 2024-11-05 22:04:14 +04:00
Zonghang Li
6657885816 fix swap detect on linux 2024-11-05 21:57:09 +04:00
Lizonghang
2d447266e9 add swap capacity test 2024-11-05 21:42:45 +04:00
Lizonghang
9eed6b14bf add disk read speed test 2024-11-05 21:12:02 +04:00
Lizonghang
9cd66f2145 add profiler 2024-11-05 20:29:09 +04:00
Lizonghang
766ec7862b test 2024-11-05 17:22:24 +04:00
Lizonghang
976a4c3534 fix cuda support 2024-11-04 11:00:01 +04:00
Lizonghang
9d6a6845ac fix cuda support 2024-11-03 12:02:30 +04:00
Lizonghang
684b2ac05b fix cuda support 2024-11-03 11:19:50 +04:00
Lizonghang
a83f577c63 reset backend_embd and out_embd as input tensors 2024-11-02 20:40:21 +04:00
Lizonghang
7aa771e701 set backend_embd and out_embd on the device of their adjacent nodes 2024-11-02 09:41:40 +04:00
Lizonghang
bfc3f9e185 split input as a separate subgraph 2024-10-30 23:40:13 +04:00
Lizonghang
64291ad70c fix next_sub_gf 2024-10-28 11:03:46 +04:00
Lizonghang
925ea39382 finetune prefetch 2024-10-28 10:58:48 +04:00
Lizonghang
76a7fc7527 support different window sizes 2024-10-26 12:34:14 +04:00
Lizonghang
5685cb87ed delete .github 2024-10-25 17:32:13 +04:00
Lizonghang
c97ea10617 add mmap prefetch and unloading 2024-10-25 16:33:56 +04:00
Lizonghang
ba5117581e init 2024-10-23 14:29:14 +04:00
Lizonghang
76d67387c0 init 2024-10-23 09:51:30 +04:00
Lizonghang
2a01ff5fb1 init 2024-10-23 09:42:32 +04:00
Diego Devesa
6374743747
ggml : add backend registry / device interfaces to BLAS backend (#9752) 
* ggml : add backend registry / device interfaces to BLAS backend

* fix mmap usage when using host buffers
 2024-10-07 21:55:08 +02:00
Andrew Minh Nguyen
f1af42fa8c
Update building for Android (#9672) 
* docs : clarify building Android on Termux

* docs : update building Android on Termux

* docs : add cross-compiling for Android

* cmake : link dl explicitly for Android
 2024-10-07 09:37:31 -07:00
Georgi Gerganov
6279dac039
flake.lock: Update (#9753) 
Flake lock file updates:

• Updated input 'flake-parts':
    'github:hercules-ci/flake-parts/bcef6817a8b2aa20a5a6dbb19b43e63c5bf8619a?narHash=sha256-HO4zgY0ekfwO5bX0QH/3kJ/h4KvUDFZg8YpkNwIbg1U%3D' (2024-09-12)
  → 'github:hercules-ci/flake-parts/3d04084d54bedc3d6b8b736c70ef449225c361b1?narHash=sha256-K5ZLCyfO/Zj9mPFldf3iwS6oZStJcU4tSpiXTMYaaL0%3D' (2024-10-01)
• Updated input 'flake-parts/nixpkgs-lib':
    '356624c120.tar.gz?narHash=sha256-Ss8QWLXdr2JCBPcYChJhz4xJm%2Bh/xjl4G0c0XlP6a74%3D' (2024-09-01)
  → 'fb192fec7c.tar.gz?narHash=sha256-0xHYkMkeLVQAMa7gvkddbPqpxph%2BhDzdu1XdGPJR%2BOs%3D' (2024-10-01)
• Updated input 'nixpkgs':
    'github:NixOS/nixpkgs/1925c603f17fc89f4c8f6bf6f631a802ad85d784?narHash=sha256-J%2BPeFKSDV%2BpHL7ukkfpVzCOO7mBSrrpJ3svwBFABbhI%3D' (2024-09-26)
  → 'github:NixOS/nixpkgs/bc947f541ae55e999ffdb4013441347d83b00feb?narHash=sha256-NOiTvBbRLIOe5F6RbHaAh6%2B%2BBNjsb149fGZd1T4%2BKBg%3D' (2024-10-04)

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
 2024-10-07 09:35:42 -07:00