55 commits

Author	SHA1	Message	Date
Lizonghang	a79ab3dde3	device_disk_access_delay: fix delay calculation when physical memory is busy (all mapped tensors will be reloaded)	2024-12-06 16:41:55 +04:00
Lizonghang	f1c1d1b929	add support for Q5_K and fix byte count for Q6_K	2024-12-06 07:59:45 +04:00
Lizonghang	e6f4c009ab	device_disk_access_delay: add delay for loading one row from lookup table	2024-12-05 20:38:28 +04:00
Lizonghang	3c8dbd11fc	device_compute_delay: compatible with the shared memory in Metal	2024-12-05 20:02:13 +04:00
Zonghang Li	5a99c5894d	update test model, enable warm-up and sched	2024-12-05 09:52:16 +04:00
Lizonghang	b9538f048a	add sched in device_flops but disabled by default	2024-12-04 23:54:57 +04:00
Zonghang Li	7521e532c4	device_memory_bw: simulate cache-friendly block access and multi-threading	2024-12-04 15:36:59 +04:00
Zonghang Li	44b4718c8b	recover device_memory_access_delay	2024-12-04 12:29:24 +04:00
Zonghang Li	89aad7880e	fix sysctlbyname on Linux and Win	2024-12-04 12:24:07 +04:00
Lizonghang	585864b05a	add inactive_memory in Mac UMA	2024-12-04 11:56:09 +04:00
Lizonghang	99d48157a0	compatible with Mac ARM64 and x86_64 arch, and ignore memory access delay	2024-12-04 11:33:46 +04:00
Zonghang Li	6d7801de87	fix available_memory calculation	2024-12-04 10:36:43 +04:00
Lizonghang	75a86e4eb1	use internal_mem - purgeable_mem to calculate the available_mem	2024-12-04 10:24:54 +04:00
Lizonghang	bd5e034746	set fio engine to sync if posixaio not loadable	2024-12-04 00:55:50 +04:00
Lizonghang	508c8ae83c	set fio engine to sync if posixaio not loadable	2024-12-04 00:52:01 +04:00
Lizonghang	c3d708862c	use disk read_rnd_bw by default	2024-12-04 00:37:59 +04:00
Lizonghang	7cd4936d58	rename to CPU RAM Read BW	2024-12-04 00:04:22 +04:00
Lizonghang	dc03b6216f	set default page size to 4KB if not available from system	2024-12-03 23:25:47 +04:00
Lizonghang	74dbec5086	set default readahead size to 128KB if permission is needed	2024-12-03 22:55:16 +04:00
Zonghang Li	1a7237636e	add cgroup version detect	2024-11-30 09:48:19 +04:00
Zonghang Li	3074763ed4	fix memory detect in docker container	2024-11-30 09:36:01 +04:00
Zonghang Li	eb25858e87	remove cpus_allowed_policy in fio	2024-11-29 22:16:32 +04:00
Zonghang Li	7e4bb65eab	fix bugs on cpu	2024-11-29 22:06:02 +04:00
Zonghang Li	81fd77033e	add gpu support in device_memory_access_delay	2024-11-29 21:56:01 +04:00
Lizonghang	6f54a12c7d	add gpu support in llama_model_kvcache_size and llama_model_compute_buf_size	2024-11-29 21:06:32 +04:00
Lizonghang	f8e9dc2713	add GPU support in device_compute_delay and device_disk_access_delay	2024-11-29 20:21:22 +04:00
Lizonghang	68ecabc8c3	add cpu_read_ram_bw, metal_read_vram_bw, cuda_read_vram_bw	2024-11-29 19:04:53 +04:00
Lizonghang	ce45587ea9	correct GB to GiB	2024-11-29 11:20:19 +04:00
Lizonghang	0f73d12247	decrease compute buf from available memory	2024-11-29 11:15:54 +04:00
Lizonghang	45a1e55eec	reduce kv cache from available memory	2024-11-28 20:21:21 +04:00
Lizonghang	9858d90ce4	get system readahead size automatically	2024-11-28 16:18:41 +04:00
Lizonghang	740f7f0b95	use multithread disk r/w test	2024-11-27 22:14:17 +04:00
Lizonghang	f7507ec20b	fix disk r/w test, add disk access latency, and correct units (GB, GiB)	2024-11-27 21:36:12 +04:00
Lizonghang	0a91ad3edc	fix cuda compatibility errors	2024-11-26 22:35:58 +04:00
Zonghang Li	f78c437172	add device_inp_embd_delay test, device_memory_bw test, device_cuda_memory_bw test,	2024-11-26 22:28:02 +04:00
Lizonghang	a7a95b53fe	add q80xf32 and count_n_params	2024-11-24 23:11:12 +04:00
Lizonghang	3fe00a16a0	count model flops for f32xf32, f16xf32, q4kxf32, q6kxf32	2024-11-24 13:13:32 +04:00
Lizonghang	a5ba34169a	add f32, f16, q4k_f32, q6k_f32 flops test and fix duplicate inp_embd in subgraphs	2024-11-23 21:36:34 +04:00
Zonghang Li	7ee1423006	add model_flops	2024-11-21 20:06:16 +04:00
Zonghang Li	80f6b72e71	remove device_flops from profiler api	2024-11-21 08:37:57 +04:00
Lizonghang	10f6f92c7e	add f32, f16, q8, q4k speed test for cuda	2024-11-10 23:41:13 +04:00
Lizonghang	f4260bb346	add device_flops() for cpu, metal, and cuda	2024-11-10 23:11:05 +04:00
Lizonghang	5fae6ac36f	add cpu flops test	2024-11-09 20:53:42 +04:00
Lizonghang	2bd4d03aa8	add automatic layer window size assignment workflow	2024-11-08 18:21:03 +04:00
Lizonghang	53cb3a6069	synchronize device info	2024-11-07 22:02:01 +04:00
Lizonghang	ef7fdf70cc	add LLAMA_API llama_profile_device	2024-11-07 09:30:39 +04:00
Lizonghang	407c71ae52	add cpu and gpu profile	2024-11-06 20:42:28 +04:00
Lizonghang	4e1be1065d	add memory speed test	2024-11-06 10:57:30 +04:00
Zonghang Li	9a03b52785	fix device get name on linux	2024-11-05 22:07:09 +04:00
Lizonghang	a7f3d917a1	add device get name	2024-11-05 22:04:14 +04:00