vrr/koboldcpp
2
0
Fork 0
mirror of https://github.com/LostRuins/koboldcpp.git synced 2025-09-10 17:14:36 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
2556 commits 3 branches 196 tags 386 MiB
Commit graph

317 commits

Author SHA1 Message Date
Concedo
ea81eae189 cleanup, up ver (+1 squashed commits) 
Squashed commits:

[1ea303d6] cleanup , up ver (+1 squashed commits)

Squashed commits:

[79f09b22] cleanup
 2023-11-05 22:49:23 +08:00
YellowRoseCx
e2e5fe56a8
KCPP Fetches AMD ROCm Memory without a stick, CC_TURING Gets the Boot, koboldcpp_hipblas.dll Talks To The Hand, and hipBLAS Compiler Finds Its Independence! (#517) 
* AMD ROCm memory fetching and max mem setting

* Update .gitignore with koboldcpp_hipblas.dll

* Update CMakeLists.txt remove CC_TURING for AMD

* separate hipBLAS compiler, update MMV_Y, move CXX/CC print

separate hipBLAS compiler, update MMV_Y value, move the section that prints CXX and CC compiler name
 2023-11-05 22:23:18 +08:00
Concedo
5e5be717c3 fix for removing inaccessible backends in gui 2023-11-05 10:12:12 +08:00
Concedo
1e7088a80b autopick cublas in gui if possible, better layer picking logic 2023-11-05 01:35:27 +08:00
Concedo
135001abc4 try to make the tunnel more reliable 2023-11-04 09:18:19 +08:00
Concedo
36f43ae834 syntax correction 2023-11-04 00:03:45 +08:00
Concedo
373c20ad51 print error log if tunnel fails 2023-11-03 23:48:21 +08:00
Concedo
879061c5d5 noavx2 clblast selector 2023-11-02 23:13:16 +08:00
Concedo
b0c7b88eac try fix clouflare tunnel (+2 squashed commit) 
Squashed commit:

[87d96bf2] update remote option

[c30bc909] updated fixed colab (+1 squashed commits)

Squashed commits:

[97b77563] updated fixed colab (+2 squashed commit)

Squashed commit:

[d851b04c] replaced cloudflare manual dl with remotetunnel in colab

[90ff1790] updated lite
 2023-11-02 22:27:35 +08:00
Concedo
fca7a4c054 added noavx2 model for clblast (+1 squashed commits) 
Squashed commits:

[291ecae6] added noavx2 mode for clblast (+1 squashed commits)

Squashed commits:

[562bc872] wip adding noavx2 cl
 2023-11-02 15:22:34 +08:00
Concedo
82267e5e69 switched back to clinfo since it's possibly more cross platform and can get memory vals easily 2023-11-02 14:12:05 +08:00
Concedo
21588cefd4 tunnel code done (+1 squashed commits) 
Squashed commits:

[b4bc7d20] wip integration of trycloudflare
 2023-11-01 23:28:23 +08:00
Concedo
3b227fc704 automatic gpu layer detection 2023-11-01 20:55:26 +08:00
Concedo
b395dbf6f5 wip layer calculator 2023-11-01 20:04:10 +08:00
Concedo
ae2cd56de8 kobold integration of min_p sampler (+1 squashed commits) 
Squashed commits:

[8ad2e349] kobold integration for min_p sampler
 2023-11-01 19:08:45 +08:00
Concedo
df7e757d40 windows: added simpleclinfo, which helps determine clblast platform and device on windows 2023-11-01 18:10:35 +08:00
Concedo
f3690ba6d2 shifting enabled by default 2023-10-31 21:41:57 +08:00
Concedo
61c395833d context shifting is still buggy 2023-10-30 16:25:01 +08:00
Concedo
7f5d1b2fc6 slider error 2023-10-30 00:02:38 +08:00
Concedo
7924592a83 context shift feature done 2023-10-29 18:21:39 +08:00
Concedo
09c74ea046 include content-length 2023-10-28 14:24:37 +08:00
Concedo
15f525c580 revamped smart context for llama models 2023-10-28 12:59:08 +08:00
Concedo
c2f675133d support for abort without crash on disconnect 2023-10-27 15:27:17 +08:00
Concedo
aed05e5565 todo: troubleshoot sse with multiuser 2023-10-27 00:21:52 +08:00
Concedo
5db89b90b7 Merge branch 'master' into concedo_experimental 
# Conflicts:
#	.gitignore
#	CMakeLists.txt
#	Makefile
#	README.md
#	build.zig
#	ggml-opencl.cpp
#	tests/CMakeLists.txt
#	tests/test-double-float.cpp
#	tests/test-sampling.cpp
 2023-10-25 23:58:15 +08:00
Concedo
98d1dba256 tighten timings 2023-10-25 20:44:20 +08:00
Concedo
cff75061fe fixed some old models failing due to tokenizer changes, update lite (+1 squashed commits) 
Squashed commits:

[9dee81ec] fixed some old models failing due to tokenizer changes, update lite tooltip (+3 squashed commit)

Squashed commit:

[5ab95a79] fixes

[a561d5e2] fixed some old models failing due to tokenizer changes

[95e65daf] lite updates
 2023-10-22 11:04:59 +08:00
Concedo
6fa681b692 fixed a race condition with SSE streaming 2023-10-20 22:01:09 +08:00
Concedo
4382e51719 updated lite and default horde ctx amount 2023-10-19 22:49:59 +08:00
Concedo
6f8fe88f10 fix for lite (+5 squashed commit) 
Squashed commit:

[f9ce9855] catch more exceptions

[8cdaf149] tweaked horde worker timeouts, updated lite

[619ebef4] fixed abort no response if failed

[a54a66a2] fixed time overflow

[9affdc3e] updated lite
 2023-10-17 23:04:32 +08:00
Concedo
643902fbbb fixed tensor split save and load 2023-10-13 10:07:22 +08:00
Concedo
7e2f714c9c tensor split only for cuda 2023-10-12 17:01:52 +08:00
Alexander Abushady
11b8f97c1e
Tensor split UI (#471) 
* update .gitignore

Remove .idea folder created by Jet Brains products.

* Front end, and partial backe-end

Tensor Split pulled in, shows in console, then not respected on model load.

* UI Tweak + Tensor Split Fix

Made Tensor Flow input match similar boxes around it. Also, fixed Tensor Split to populate the correct argument.

* Changed int to float for tensor split

Accidentally set int, needed to be float when setting tensor split args
 2023-10-12 16:50:21 +08:00
Concedo
8be043ee38 more horde optimizations 2023-10-12 16:20:52 +08:00
Concedo
8d1cd512e2 missed a flag 2023-10-12 15:00:51 +08:00
Concedo
c6fe820357 improve cors and header handling 2023-10-12 14:53:39 +08:00
Concedo
f604cffdce multiuser racer bugfix 2023-10-12 13:39:12 +08:00
Concedo
a003e3c348 horde auto recovery 2023-10-12 00:57:32 +08:00
Concedo
d74eab0e63 actually for this round, do not include deprecated params. i dont want to have to deal with them (+2 squashed commit) 
Squashed commit:

[df2691c2] show context limit

[7c74f52a] prevent old scripts from crashing
 2023-10-10 19:20:33 +08:00
YellowRoseCx
1b25b21655
Merge pull request #27 from one-lithe-rune/allow-sdk-dll-loading - Allow use of hip SDK (if installed) dlls on windows (#470) 
* If the rocm/hip sdk is installed on windows, then include the sdk
as a potential location to load the hipBlas/rocBlas .dlls from. This
allows running koboldcpp.py directly with python after building
work on windows without having to build the .exe and run that or
copy .dlls around.

Co-authored-by: one-lithe-rune <skapusniak@lithe-runes.com>
 2023-10-10 17:16:33 +08:00
Concedo
f288c6b5e3 Merge branch 'master' into concedo_experimental 
# Conflicts:
#	CMakeLists.txt
#	Makefile
#	build.zig
#	scripts/sync-ggml.sh
 2023-10-10 00:09:46 +08:00
Matěj Štágl
96e9539f05
OpenAI compat API adapter (#466) 
* feat: oai-adapter

* simplify optional adapter for instruct start and end tags

---------

Co-authored-by: Concedo <39025047+LostRuins@users.noreply.github.com>
 2023-10-09 23:24:48 +08:00
Concedo
4e5b6293ab adjust streaming timings 2023-10-08 23:12:45 +08:00
Concedo
a2b8473354 force flush sse 2023-10-08 15:12:07 +08:00
Concedo
07a114de63 force debugmode to be indicated on horde, allow 64k context for gguf 2023-10-07 10:23:33 +08:00
Concedo
120695ddf7 add update link 2023-10-07 01:33:18 +08:00
Concedo
2a36c85558 abort has multiuser support via genkey too 2023-10-06 23:27:00 +08:00
Concedo
1d1232ffbc show horde job count 2023-10-06 18:42:59 +08:00
Concedo
efd0567f10 Merge branch 'concedo' into concedo_experimental 
# Conflicts:
#	koboldcpp.py
 2023-10-06 11:22:01 +08:00
grawity
9d0dd7ab11
avoid leaving a zombie process for --onready (#462) 
Popen() needs to be used with 'with' or have .wait() called or be
destroyed, otherwise there is a zombie child that sticks around until
the object is GC'd.
 2023-10-06 11:06:37 +08:00