vrr/koboldcpp
2
0
Fork 0
mirror of https://github.com/LostRuins/koboldcpp.git synced 2025-09-10 17:14:36 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
4938 commits 3 branches 196 tags 386 MiB
Commit graph

565 commits

Author SHA1 Message Date
Concedo
4b00916ac7 Merge branch 'master' into concedo_experimental 
# Conflicts:
#	.dockerignore
#	.github/workflows/build.yml
#	CMakeLists.txt
#	Makefile
#	README.md
#	flake.lock
#	flake.nix
#	tests/CMakeLists.txt
 2023-08-28 14:19:05 +08:00
Concedo
b95a4ccb22 added a token counting endpoint, set mmq as default 2023-08-24 20:41:49 +08:00
Concedo
598d4d89ab fix for config file loading. from kcpp settings file 2023-08-24 15:45:33 +08:00
Concedo
a3b9949626 Merge remote-tracking branch 'pop/add_config_arg' into concedo_experimental 2023-08-24 15:22:17 +08:00
Pontus Mårdnäs
2d86b2e219 Add --config argument 2023-08-21 23:46:56 +02:00
Concedo
36b0c5b398 fix for incorrect missing backends displayed 2023-08-17 22:45:49 +08:00
Concedo
469d70be45 add support for precompiled binaries, used as a fallback 2023-08-15 13:49:05 +08:00
Concedo
dae9dffa6a rename koboldcpp.dll to koboldcpp_default.dll 2023-08-11 14:54:27 +08:00
Concedo
886f4eed79 updated lite, up ver, remove bell 2023-08-10 22:01:33 +08:00
Concedo
c5f5209d37 globalize args 2023-08-10 16:30:02 +08:00
Concedo
f8376c7e61 up ver, fixed compile (+1 squashed commits) 
Squashed commits:

[ca51aa9e] up ver
 2023-08-09 21:31:24 +08:00
Concedo
90058d96b0 sleep longer before exit 2023-08-09 15:28:07 +08:00
Concedo
19cf2a8663 add idle field and up ver 2023-08-09 12:42:59 +08:00
Concedo
159ad9269d up ver, set the cuda pool malloc lookahead back to 5% instead of 2% (+1 squashed commits) 
Squashed commits:

[e0f65278] up ver, set the cuda pool malloc lookahead back to 5% instead of 2%
 2023-08-09 12:06:42 +08:00
Concedo
793cfd136c fixed 70B detection again, try fix horde issues, fixed lite unicode issue, fixed cmake for cuda 2023-08-09 01:05:00 +08:00
Concedo
cae6a847ad cuda free only for non mmq (+2 squashed commit) 
Squashed commit:

[3aca763a] only cuda free for non mmq

[e69a8c9f] revert to pool alloc to try again
 2023-08-07 17:12:05 +08:00
Concedo
e99416cdfe blasbatchsize 2023-08-06 17:47:59 +08:00
Concedo
bcfdd0e662 fixed bbs -1 and allow bbs = 2048 2023-08-06 17:47:05 +08:00
Concedo
18bb0ab127 up ver, support 16k ctx 2023-08-04 21:47:17 +08:00
Concedo
f0764c6cfb fix indentation, increase server thread count 2023-08-04 10:29:56 +08:00
Concedo
d09e54aad1 Merge remote-tracking branch 'duncan/api-stream-double-write-fix' into concedo_experimental 2023-08-04 10:22:53 +08:00
duncannah
63ec711a70
fix: still send full result after streaming 2023-08-03 14:35:43 +02:00
Concedo
4709545c06 Merge remote-tracking branch 'duncan/api-stream-double-write-fix' into concedo_experimental 2023-08-03 12:52:43 +08:00
Concedo
ba2040d1df compile fix for ARM NEON 2023-08-03 12:52:06 +08:00
duncannah
9281c2801f
fix: don't send headers twice when streaming 2023-08-02 23:42:43 +02:00
Concedo
46682e5cb3 added mmq launch flag 2023-08-01 17:57:13 +08:00
Concedo
84ce184c4f layout 2023-07-31 17:33:31 +08:00
YellowRoseCx
f27972777f
correct semantic error in import_vars (#355) 
* Hide unavailable backends & Add tooltip over backend count

Hides unavailable backends from the user and if the program is launched without any backends made, it shows an error message to them stating no backends were found and to make them using the 'make' command

Add tooltip when hovering over backend count label

hovering over the new label that shows the backend count will explain what the numbers are, and show the users which backends are not available or built

* add some code comments

* hide "missing" if all are built

move tooltip functions to helper functions section. hides the string "Missing: ..." from showing if all backends are available
" if len(runopts)==6 else + "

* small typo fix

* remove wrongly added leftover device choosing code

* fix labels

* move tooltip to function

* import vars logic fix

---------

Co-authored-by: Concedo <39025047+LostRuins@users.noreply.github.com>
 2023-07-31 15:51:35 +08:00
Concedo
5ea5d19d6a SSE emoji fix 2023-07-30 22:31:20 +08:00
Concedo
90a37d63d5 up ver, added warning for max context 2023-07-30 18:07:14 +08:00
YellowRoseCx
c8af65760f
Hide unavailable backends & Add tooltip over backend count (#352) 
* Hide unavailable backends & Add tooltip over backend count

Hides unavailable backends from the user and if the program is launched without any backends made, it shows an error message to them stating no backends were found and to make them using the 'make' command

Add tooltip when hovering over backend count label

hovering over the new label that shows the backend count will explain what the numbers are, and show the users which backends are not available or built

* add some code comments

* hide "missing" if all are built

move tooltip functions to helper functions section. hides the string "Missing: ..." from showing if all backends are available
" if len(runopts)==6 else + "

* small typo fix

* remove wrongly added leftover device choosing code

* fix labels

* move tooltip to function

---------

Co-authored-by: Concedo <39025047+LostRuins@users.noreply.github.com>
 2023-07-30 17:50:55 +08:00
Concedo
45456fa6ca switch noavx2 to not use openblas, as it has incompatible instructions 2023-07-30 16:47:33 +08:00
Concedo
23825abee1 fix wrong key 2023-07-30 14:30:46 +08:00
Concedo
9589d52079 added help link 2023-07-29 17:33:15 +08:00
Concedo
e4b42e5b15 fixed gui bugs 2023-07-29 11:15:57 +08:00
Concedo
94e0a06daf updated lite, up ver (+1 squashed commits) 
Squashed commits:

[7d6520f] updated lite, up ver
 2023-07-26 11:03:17 +08:00
Concedo
66e4b5141e fix horde worker host and client agent 2023-07-25 18:18:41 +08:00
Concedo
d8d2449bfb better label (+1 squashed commits) 
Squashed commits:

[f573b2c] cuda 3 target arch
 2023-07-24 23:07:31 +08:00
Concedo
7555dae4cc ditch advanced subparsers 2023-07-24 22:40:36 +08:00
Concedo
825e34baa3 default horde name and better handling for horde (+3 squashed commit) 
Squashed commit:

[fadfa60] better idle handling for horde worker

[a3971e6] updated lite

[2ca2b79] seems to not generate rubbish
 2023-07-24 18:41:41 +08:00
Concedo
c7136f03d9 added support for tensor_split parameter as an advanced parameter. 2023-07-24 17:16:19 +08:00
Concedo
280abaf029 added stop reason in the perf endpoint 2023-07-24 11:55:35 +08:00
Concedo
0cca0726fe reduce number of retries, fixed maxlength > maxctx bug 2023-07-23 09:59:34 +08:00
Concedo
fa0270df7c added some checks to skip generation if busy 2023-07-22 23:10:04 +08:00
Concedo
2807d98fd4 touchup (+2 squashed commit) 
Squashed commit:

[8b06458] fixed broken param order

[7eabdc0] very broken, do not use
 2023-07-22 22:57:56 +08:00
Concedo
3aec3038d4 bump scratch buffers 2023-07-22 18:12:18 +08:00
Concedo
52c5856a08 auto populate horde model name 2023-07-22 16:03:12 +08:00
Concedo
dd3f8dabed updated cluster to horde.koboldai.net 2023-07-22 12:42:40 +08:00
Concedo
236d0e8955 add tip about using other workers 2023-07-22 12:29:22 +08:00
Concedo
701bf0a6cd reduce sleep time between jobs 2023-07-22 11:56:43 +08:00