4090 commits

This branch

This branch

All branches

Author	SHA1	Message	Date
Lizonghang	64c4a47980	fix bugs and warnings	2025-02-15 17:33:03 +04:00
Zonghang Li	630556bc16	fix default allocation strategy to avoid OOM	2025-02-15 17:23:19 +04:00
Lizonghang	fdfaaecd5e	disable device profiler in standalone mode	2025-02-12 17:12:30 +04:00
Lizonghang	c84f9d29fe	use arg prefetch and remove arg unload	2025-02-12 17:04:41 +04:00
Lizonghang	708b1d8c89	disable force fetching	2025-02-12 16:55:44 +04:00
Lizonghang	ea0e655a8b	disable force feteching	2025-02-12 16:55:21 +04:00
Lizonghang	b163918b46	disable prefetch in standalone mode	2025-02-12 00:17:33 +04:00
Lizonghang	6a50d494d2	increase prefetch dense	2025-02-11 17:25:06 +04:00
Lizonghang	65ad14140a	do not check loaded tensors due to increased latency	2025-02-11 17:10:11 +04:00
Lizonghang	3dd3138207	ignore tensors already in page cache when prefetching	2025-02-11 17:00:17 +04:00
Lizonghang	24974a488c	assume 10% of active pages can be compressed on macOS UMA	2025-02-11 11:06:33 +04:00
Zonghang Li	261c88f058	skip tensors on CUDA in manage_graph_tensors	2025-02-11 09:49:17 +04:00
Zonghang Li	c4c6a642fc	manage_graph_tensors: fix segment prefetch	2025-02-08 22:44:38 +04:00
Lizonghang	d2bc5cd502	add pid as suffix to avoid conflicts with other processes	2025-02-07 10:29:22 +04:00
Lizonghang	8e41362af0	fix prefetch	2025-02-04 17:38:41 +04:00
Lizonghang	ec73e239c9	use 80% available mem as a conservative estimate	2025-02-03 18:10:05 +04:00
Lizonghang	64089236eb	fix latency estimation in set m1	2025-02-03 07:56:02 +04:00
Lizonghang	83b3d01844	fix delay estimation on macos	2025-02-01 10:37:56 +04:00
Lizonghang	215151918f	fix gpu mem limit	2025-01-31 18:52:13 +04:00
Lizonghang	17cd8ba618	reverse 300MiB for Metal kernel	2025-01-31 16:24:44 +04:00
Lizonghang	dd632ee6df	ignore the first 5 evals due to preheat	2025-01-31 08:53:51 +04:00
Lizonghang	fdecd4b54c	more active pages can be compressed	2025-01-30 23:17:07 +04:00
Lizonghang	2bc7a56790	fix available mem estimation in termux	2025-01-30 21:23:05 +04:00
Lizonghang	b680cb74fe	set POSIX_MADV_WILLNEED for the next subgraph	2025-01-30 13:29:34 +04:00
Lizonghang	f9b4c46b74	ignore the first eval to make time test more accurate	2025-01-30 11:12:26 +04:00
Lizonghang	cd758247e6	consider active pages compression in macos available memory estimation	2025-01-29 20:33:13 +04:00
Lizonghang	27c996835d	fix undeclared identifier get_page_size	2025-01-29 19:59:02 +04:00
Lizonghang	4b616baed4	fix macos x86_64 available mem estimation	2025-01-29 19:57:06 +04:00
Lizonghang	849b47ccd0	fix auto schedule logic	2025-01-29 13:13:37 +04:00
Lizonghang	e7c6b830e6	fix auto schedule logic	2025-01-29 11:15:45 +04:00
Lizonghang	631daadd92	test	2025-01-28 16:36:47 +04:00
Zonghang Li	36f353e374	check env path before calling fio to ensure we can find it	2025-01-28 13:06:08 +04:00
Zonghang Li	2d649c808d	add /usr/local/bin on darwin	2025-01-28 13:05:25 +04:00
Lizonghang	2934cf3e8e	reserve 200 mib for internal gpu usage	2025-01-27 22:14:12 +04:00
Lizonghang	1ca9e7974b	device_os returns Linux if in Termux	2025-01-27 11:14:21 +04:00
Lizonghang	1e2b934d69	add bounds n[m]<=0 for devices without GPUs	2025-01-27 11:13:09 +04:00
Lizonghang	ac5d63b09e	add explaination for why the output layer weights should be kept in metal shared memory	2025-01-25 23:51:16 +04:00
Lizonghang	1ca9a43bd1	keep the output layer weights in shared memory by default	2025-01-25 23:31:43 +04:00
Lizonghang	f3dd5776eb	fix kappa and memory bounds, account for look-up table and input/output layer delay	2025-01-25 22:31:40 +04:00
Lizonghang	9e4ba4f06a	fix w init error	2025-01-24 20:26:18 +04:00
Lizonghang	4948b1004c	fix device_name() to get device name from host, not from termux	2025-01-24 16:56:23 +04:00
Lizonghang	1c0087e919	rename arg --keep-inp-out-in-metal to --keep-out-in-metal	2025-01-23 23:17:06 +04:00
Lizonghang	5fcf020cfb	use the recommended working set size as metal mem bound, rather than the total physical mem	2025-01-23 23:04:40 +04:00
Lizonghang	fb05f80f89	remove token_embd from metal mem	2025-01-23 16:58:25 +04:00
Lizonghang	92f00303d5	set reserved mem to 100MiB	2025-01-23 16:58:11 +04:00
Lizonghang	38e4a3eaa0	add device_macos_swappable_memory	2025-01-23 16:09:04 +04:00
Lizonghang	78a544d716	add metal mem limit	2025-01-23 16:08:52 +04:00
Zonghang Li	33429ec4e1	add option --keep-inp-out-in-metal	2025-01-22 11:25:09 +04:00
Lizonghang	facb4ea736	add option --keep-inp-out-in-metal and fix bugs in unmap	2025-01-22 11:15:19 +04:00
Lizonghang	ce2ef9699f	fix mapping unmap_fragment error	2025-01-21 22:43:51 +04:00