308 commits

Author	SHA1	Message	Date
Concedo	66a3f4e421	added support for lora base	2023-06-10 19:29:45 +08:00
Concedo	a68fcfe738	only start a new thread when using sse	2023-06-10 19:03:41 +08:00
Concedo	43f7e40470	added extra endpoints for abort gen and polled streaming	2023-06-10 18:13:26 +08:00
Concedo	5bd9cef9fa	merging Proper SSE Token Streaming #220 with end connection fix test	2023-06-09 23:22:16 +08:00
SammCheese	57b0b53b54	fix kobold lite generation	2023-06-09 12:39:35 +02:00
SammCheese	e6231c3055	back to http.server, improved implementation	2023-06-09 12:17:55 +02:00
SammCheese	dee692a63e	compability with basic_api, change api path to /extra	2023-06-08 18:34:24 +02:00
SammCheese	b4e9e185d3	fix legacy streaming	2023-06-08 18:34:24 +02:00
SammCheese	9a8da35ec4	working streaming. TODO: fix lite	2023-06-08 18:34:23 +02:00
SammCheese	97971291e9	draft: token streaming	2023-06-08 18:34:08 +02:00
Concedo	a6a0fa338a	cleanup indentation, fixing cublas build	2023-06-08 22:40:53 +08:00
Concedo	c046db5197	lite bugfixes, buffer size changes, fixed a topk bug.	2023-06-06 22:38:25 +08:00
Concedo	79df932d0a	added dropdown for blasbatch. added capability to build avx clblast but not in default build for now	2023-06-05 22:50:21 +08:00
Concedo	9aa2d8535b	hide gpu input box when dropdown not selected, minor memory fix for neox and gptj	2023-06-04 21:47:17 +08:00
Concedo	c3c05fc33b	further cleanup, refactor renamemode to hordeconfig	2023-06-04 11:57:46 +08:00
Concedo	8bd9a3a48b	updated readme, improved simple launcher	2023-06-03 17:17:15 +08:00
Concedo	9839259b63	allow specifying the horde limit as well	2023-06-03 00:55:44 +08:00
Concedo	37659d2c4e	allow blasbatchsize -1 which disables blas, but keeps benefits like gpu offloads.	2023-06-01 22:33:50 +08:00
Concedo	49272e3c53	adjusted defaults	2023-06-01 20:03:44 +08:00
Concedo	32dada5e5f	updated lite	2023-05-31 17:52:09 +08:00
Concedo	30ff1133f5	allow users to rename models for use in horde	2023-05-29 16:01:05 +08:00
Concedo	97b39f875c	fixed fstat64 build error on mac	2023-05-29 15:50:07 +08:00
Concedo	fe63bfdb0f	Revert "allow 2048 blasbatchsize" ... This reverts commit 94dc5c2324.	2023-05-27 18:13:27 +08:00
Concedo	94dc5c2324	allow 2048 blasbatchsize	2023-05-27 17:47:18 +08:00
Concedo	abfdfb702e	added top_a sampler	2023-05-27 17:32:37 +08:00
Concedo	bd4fe936f5	cleanup sampling code	2023-05-27 11:58:39 +08:00
Concedo	8b8f2f4cf5	up ver to 1.25.1	2023-05-25 14:49:30 +08:00
Concedo	37a34deaa0	added a second pyinstaller for my own use that uses a different python version. don't use this.	2023-05-24 23:34:11 +08:00
Concedo	d04b3bbe5e	disable mmap when failsafe mode selected from GUI	2023-05-24 15:04:17 +08:00
Concedo	355007b019	added sampler seed	2023-05-23 21:52:26 +08:00
Concedo	cd4012c3ed	minor fixes to debug logging, fixed a typo, added a new failsafe mode	2023-05-23 21:31:42 +08:00
Concedo	587308a202	fixed some build errors on linux, changed icon resolution, added more error printing	2023-05-22 12:18:42 +08:00
Concedo	24127ebf98	updated lite, fixed some encoding issues	2023-05-21 17:29:00 +08:00
Concedo	c048bcfec4	remove old filever checks (+7 squashed commit) ... Squashed commit: [b72627a] new format not working [e568870] old ver works [7053b77] compile errors fixed, fixing linkers [4ae8889] add new ver [ff82dfd] file format checks [25b8aa8] refactoring type names [931063b] still merging	2023-05-21 00:15:39 +08:00
Concedo	f5e1fe46e1	up ver	2023-05-18 17:15:03 +08:00
Concedo	76dc539172	change help text	2023-05-17 18:06:47 +08:00
Concedo	e4e6994353	Not working, don't use. testing a merge	2023-05-16 12:33:24 +08:00
Concedo	fce2e7e518	up version	2023-05-15 14:53:13 +08:00
Concedo	466cd21368	test cmakefile for cublas.	2023-05-15 14:50:38 +08:00
Concedo	9cd5b9a769	up ver	2023-05-14 11:10:26 +08:00
Concedo	e05455f852	fixed wrong sized struct from legacy q8_1, fixed opencl varsize arrays	2023-05-13 23:56:08 +08:00
Concedo	b6594ab91e	do not show tokenizer warning	2023-05-13 15:48:17 +08:00
Concedo	6d87f67572	up ver	2023-05-09 17:25:46 +08:00
Concedo	b9904c3093	up ver	2023-05-08 11:13:16 +08:00
Concedo	89d70886a4	added support for setting custom context size at load time (memory allocation)	2023-05-08 11:11:25 +08:00
Concedo	ff93b394da	fixed a typo	2023-05-06 12:37:34 +08:00
Concedo	8a964e76c8	integrated mirostat as a launch parameter, works on all models	2023-05-06 00:47:17 +08:00
Concedo	851f55325a	Merge remote-tracking branch 'temp/concedo' into concedo_experimental	2023-05-05 23:55:53 +08:00
Concedo	2edbcebe27	added optional force versioning flag	2023-05-05 22:02:00 +08:00
Hendrik Langer	8131bc8b56	add new sampling algorithm mirostat	2023-05-05 13:23:47 +02:00