4196 commits

This branch

This branch

All branches

Author	SHA1	Message	Date
Lizonghang	639a33dfa5	add op GGML_OP_READ and GGML_METAL_KERNEL_TYPE_READ_VRAM	2024-11-29 17:15:19 +04:00
Lizonghang	6934d52d8a	graph->nodes[i] => node	2024-11-29 17:14:25 +04:00
Lizonghang	ce45587ea9	correct GB to GiB	2024-11-29 11:20:19 +04:00
Lizonghang	0f73d12247	decrease compute buf from available memory	2024-11-29 11:15:54 +04:00
Lizonghang	329d084061	add illustration for memory allocation of activations	2024-11-29 10:34:21 +04:00
Lizonghang	45a1e55eec	reduce kv cache from available memory	2024-11-28 20:21:21 +04:00
Lizonghang	9858d90ce4	get system readahead size automatically	2024-11-28 16:18:41 +04:00
Lizonghang	9a7bbce7ad	fix t_load_us	2024-11-28 15:55:21 +04:00
Lizonghang	740f7f0b95	use multithread disk r/w test	2024-11-27 22:14:17 +04:00
Lizonghang	f7507ec20b	fix disk r/w test, add disk access latency, and correct units (GB, GiB)	2024-11-27 21:36:12 +04:00
Lizonghang	9cd22177d0	remove arg test_file	2024-11-27 21:34:45 +04:00
Lizonghang	0a91ad3edc	fix cuda compatibility errors	2024-11-26 22:35:58 +04:00
Zonghang Li	3f008f2ad9	add op GGML_OP_READ	2024-11-26 22:28:34 +04:00
Zonghang Li	f78c437172	add device_inp_embd_delay test, device_memory_bw test, device_cuda_memory_bw test,	2024-11-26 22:28:02 +04:00
Lizonghang	a7a95b53fe	add q80xf32 and count_n_params	2024-11-24 23:11:12 +04:00
Lizonghang	3fe00a16a0	count model flops for f32xf32, f16xf32, q4kxf32, q6kxf32	2024-11-24 13:13:32 +04:00
Lizonghang	a5ba34169a	add f32, f16, q4k_f32, q6k_f32 flops test and fix duplicate inp_embd in subgraphs	2024-11-23 21:36:34 +04:00
Zonghang Li	7ee1423006	add model_flops	2024-11-21 20:06:16 +04:00
Zonghang Li	80f6b72e71	remove device_flops from profiler api	2024-11-21 08:37:57 +04:00
Lizonghang	477ecf2084	add llama_model_n_flops	2024-11-20 19:40:27 +04:00
Lizonghang	10f6f92c7e	add f32, f16, q8, q4k speed test for cuda	2024-11-10 23:41:13 +04:00
Lizonghang	f4260bb346	add device_flops() for cpu, metal, and cuda	2024-11-10 23:11:05 +04:00
Lizonghang	5fae6ac36f	add cpu flops test	2024-11-09 20:53:42 +04:00
Lizonghang	2bd4d03aa8	add automatic layer window size assignment workflow	2024-11-08 18:21:03 +04:00
Lizonghang	53cb3a6069	synchronize device info	2024-11-07 22:02:01 +04:00
Lizonghang	ef7fdf70cc	add LLAMA_API llama_profile_device	2024-11-07 09:30:39 +04:00
Zonghang Li	b922418cca	convert MB to GB	2024-11-06 20:47:17 +04:00
Lizonghang	407c71ae52	add cpu and gpu profile	2024-11-06 20:42:28 +04:00
Lizonghang	4e1be1065d	add memory speed test	2024-11-06 10:57:30 +04:00
Zonghang Li	9a03b52785	fix device get name on linux	2024-11-05 22:07:09 +04:00
Lizonghang	a7f3d917a1	add device get name	2024-11-05 22:04:14 +04:00
Zonghang Li	6657885816	fix swap detect on linux	2024-11-05 21:57:09 +04:00
Lizonghang	2d447266e9	add swap capacity test	2024-11-05 21:42:45 +04:00
Lizonghang	9eed6b14bf	add disk read speed test	2024-11-05 21:12:02 +04:00
Lizonghang	9cd66f2145	add profiler	2024-11-05 20:29:09 +04:00
Lizonghang	766ec7862b	test	2024-11-05 17:22:24 +04:00
Lizonghang	976a4c3534	fix cuda support	2024-11-04 11:00:01 +04:00
Lizonghang	9d6a6845ac	fix cuda support	2024-11-03 12:02:30 +04:00
Lizonghang	684b2ac05b	fix cuda support	2024-11-03 11:19:50 +04:00
Lizonghang	a83f577c63	reset backend_embd and out_embd as input tensors	2024-11-02 20:40:21 +04:00
Lizonghang	7aa771e701	set backend_embd and out_embd on the device of their adjacent nodes	2024-11-02 09:41:40 +04:00
Lizonghang	bfc3f9e185	split input as a separate subgraph	2024-10-30 23:40:13 +04:00
Lizonghang	64291ad70c	fix next_sub_gf	2024-10-28 11:03:46 +04:00
Lizonghang	925ea39382	finetune prefetch	2024-10-28 10:58:48 +04:00
Lizonghang	76a7fc7527	support different window sizes	2024-10-26 12:34:14 +04:00
Lizonghang	5685cb87ed	delete .github	2024-10-25 17:32:13 +04:00
Lizonghang	c97ea10617	add mmap prefetch and unloading	2024-10-25 16:33:56 +04:00
Lizonghang	ba5117581e	init	2024-10-23 14:29:14 +04:00
Lizonghang	76d67387c0	init	2024-10-23 09:51:30 +04:00
Lizonghang	2a01ff5fb1	init	2024-10-23 09:42:32 +04:00