Concedo
fb3bcac368
handle memory separately for kcpp
2023-11-07 17:15:14 +08:00
Concedo
ea81eae189
cleanup, up ver (+1 squashed commits)
...
Squashed commits:
[1ea303d6] cleanup , up ver (+1 squashed commits)
Squashed commits:
[79f09b22] cleanup
2023-11-05 22:49:23 +08:00
YellowRoseCx
e2e5fe56a8
KCPP Fetches AMD ROCm Memory without a stick, CC_TURING Gets the Boot, koboldcpp_hipblas.dll Talks To The Hand, and hipBLAS Compiler Finds Its Independence! ( #517 )
...
* AMD ROCm memory fetching and max mem setting
* Update .gitignore with koboldcpp_hipblas.dll
* Update CMakeLists.txt remove CC_TURING for AMD
* separate hipBLAS compiler, update MMV_Y, move CXX/CC print
separate hipBLAS compiler, update MMV_Y value, move the section that prints CXX and CC compiler name
2023-11-05 22:23:18 +08:00
Concedo
5e5be717c3
fix for removing inaccessible backends in gui
2023-11-05 10:12:12 +08:00
Concedo
1e7088a80b
autopick cublas in gui if possible, better layer picking logic
2023-11-05 01:35:27 +08:00
Concedo
135001abc4
try to make the tunnel more reliable
2023-11-04 09:18:19 +08:00
Concedo
36f43ae834
syntax correction
2023-11-04 00:03:45 +08:00
Concedo
373c20ad51
print error log if tunnel fails
2023-11-03 23:48:21 +08:00
Concedo
879061c5d5
noavx2 clblast selector
2023-11-02 23:13:16 +08:00
Concedo
b0c7b88eac
try fix clouflare tunnel (+2 squashed commit)
...
Squashed commit:
[87d96bf2] update remote option
[c30bc909] updated fixed colab (+1 squashed commits)
Squashed commits:
[97b77563] updated fixed colab (+2 squashed commit)
Squashed commit:
[d851b04c] replaced cloudflare manual dl with remotetunnel in colab
[90ff1790] updated lite
2023-11-02 22:27:35 +08:00
Concedo
fca7a4c054
added noavx2 model for clblast (+1 squashed commits)
...
Squashed commits:
[291ecae6] added noavx2 mode for clblast (+1 squashed commits)
Squashed commits:
[562bc872] wip adding noavx2 cl
2023-11-02 15:22:34 +08:00
Concedo
82267e5e69
switched back to clinfo since it's possibly more cross platform and can get memory vals easily
2023-11-02 14:12:05 +08:00
Concedo
21588cefd4
tunnel code done (+1 squashed commits)
...
Squashed commits:
[b4bc7d20] wip integration of trycloudflare
2023-11-01 23:28:23 +08:00
Concedo
3b227fc704
automatic gpu layer detection
2023-11-01 20:55:26 +08:00
Concedo
b395dbf6f5
wip layer calculator
2023-11-01 20:04:10 +08:00
Concedo
ae2cd56de8
kobold integration of min_p sampler (+1 squashed commits)
...
Squashed commits:
[8ad2e349] kobold integration for min_p sampler
2023-11-01 19:08:45 +08:00
Concedo
df7e757d40
windows: added simpleclinfo, which helps determine clblast platform and device on windows
2023-11-01 18:10:35 +08:00
Concedo
f3690ba6d2
shifting enabled by default
2023-10-31 21:41:57 +08:00
Concedo
61c395833d
context shifting is still buggy
2023-10-30 16:25:01 +08:00
Concedo
7f5d1b2fc6
slider error
2023-10-30 00:02:38 +08:00
Concedo
7924592a83
context shift feature done
2023-10-29 18:21:39 +08:00
Concedo
09c74ea046
include content-length
2023-10-28 14:24:37 +08:00
Concedo
15f525c580
revamped smart context for llama models
2023-10-28 12:59:08 +08:00
Concedo
c2f675133d
support for abort without crash on disconnect
2023-10-27 15:27:17 +08:00
Concedo
aed05e5565
todo: troubleshoot sse with multiuser
2023-10-27 00:21:52 +08:00
Concedo
5db89b90b7
Merge branch 'master' into concedo_experimental
...
# Conflicts:
# .gitignore
# CMakeLists.txt
# Makefile
# README.md
# build.zig
# ggml-opencl.cpp
# tests/CMakeLists.txt
# tests/test-double-float.cpp
# tests/test-sampling.cpp
2023-10-25 23:58:15 +08:00
Concedo
98d1dba256
tighten timings
2023-10-25 20:44:20 +08:00
Concedo
cff75061fe
fixed some old models failing due to tokenizer changes, update lite (+1 squashed commits)
...
Squashed commits:
[9dee81ec] fixed some old models failing due to tokenizer changes, update lite tooltip (+3 squashed commit)
Squashed commit:
[5ab95a79] fixes
[a561d5e2] fixed some old models failing due to tokenizer changes
[95e65daf] lite updates
2023-10-22 11:04:59 +08:00
Concedo
6fa681b692
fixed a race condition with SSE streaming
2023-10-20 22:01:09 +08:00
Concedo
4382e51719
updated lite and default horde ctx amount
2023-10-19 22:49:59 +08:00
Concedo
6f8fe88f10
fix for lite (+5 squashed commit)
...
Squashed commit:
[f9ce9855] catch more exceptions
[8cdaf149] tweaked horde worker timeouts, updated lite
[619ebef4] fixed abort no response if failed
[a54a66a2] fixed time overflow
[9affdc3e] updated lite
2023-10-17 23:04:32 +08:00
Concedo
643902fbbb
fixed tensor split save and load
2023-10-13 10:07:22 +08:00
Concedo
7e2f714c9c
tensor split only for cuda
2023-10-12 17:01:52 +08:00
Alexander Abushady
11b8f97c1e
Tensor split UI ( #471 )
...
* update .gitignore
Remove .idea folder created by Jet Brains products.
* Front end, and partial backe-end
Tensor Split pulled in, shows in console, then not respected on model load.
* UI Tweak + Tensor Split Fix
Made Tensor Flow input match similar boxes around it. Also, fixed Tensor Split to populate the correct argument.
* Changed int to float for tensor split
Accidentally set int, needed to be float when setting tensor split args
2023-10-12 16:50:21 +08:00
Concedo
8be043ee38
more horde optimizations
2023-10-12 16:20:52 +08:00
Concedo
8d1cd512e2
missed a flag
2023-10-12 15:00:51 +08:00
Concedo
c6fe820357
improve cors and header handling
2023-10-12 14:53:39 +08:00
Concedo
f604cffdce
multiuser racer bugfix
2023-10-12 13:39:12 +08:00
Concedo
a003e3c348
horde auto recovery
2023-10-12 00:57:32 +08:00
Concedo
d74eab0e63
actually for this round, do not include deprecated params. i dont want to have to deal with them (+2 squashed commit)
...
Squashed commit:
[df2691c2] show context limit
[7c74f52a] prevent old scripts from crashing
2023-10-10 19:20:33 +08:00
YellowRoseCx
1b25b21655
Merge pull request #27 from one-lithe-rune/allow-sdk-dll-loading - Allow use of hip SDK (if installed) dlls on windows ( #470 )
...
* If the rocm/hip sdk is installed on windows, then include the sdk
as a potential location to load the hipBlas/rocBlas .dlls from. This
allows running koboldcpp.py directly with python after building
work on windows without having to build the .exe and run that or
copy .dlls around.
Co-authored-by: one-lithe-rune <skapusniak@lithe-runes.com>
2023-10-10 17:16:33 +08:00
Concedo
f288c6b5e3
Merge branch 'master' into concedo_experimental
...
# Conflicts:
# CMakeLists.txt
# Makefile
# build.zig
# scripts/sync-ggml.sh
2023-10-10 00:09:46 +08:00
Matěj Štágl
96e9539f05
OpenAI compat API adapter ( #466 )
...
* feat: oai-adapter
* simplify optional adapter for instruct start and end tags
---------
Co-authored-by: Concedo <39025047+LostRuins@users.noreply.github.com>
2023-10-09 23:24:48 +08:00
Concedo
4e5b6293ab
adjust streaming timings
2023-10-08 23:12:45 +08:00
Concedo
a2b8473354
force flush sse
2023-10-08 15:12:07 +08:00
Concedo
07a114de63
force debugmode to be indicated on horde, allow 64k context for gguf
2023-10-07 10:23:33 +08:00
Concedo
120695ddf7
add update link
2023-10-07 01:33:18 +08:00
Concedo
2a36c85558
abort has multiuser support via genkey too
2023-10-06 23:27:00 +08:00
Concedo
1d1232ffbc
show horde job count
2023-10-06 18:42:59 +08:00
Concedo
efd0567f10
Merge branch 'concedo' into concedo_experimental
...
# Conflicts:
# koboldcpp.py
2023-10-06 11:22:01 +08:00
