Concedo
0e5f16de53
reduce max ctx to fit instead of crashing
2023-11-27 19:08:54 +08:00
Concedo
2f51a6afd5
trigger quiet mode when selecting remotetunnel
2023-11-27 00:16:36 +08:00
Concedo
bffa78116d
explore quiet mode
2023-11-26 23:57:27 +08:00
Concedo
eb42c73953
revert auto rope scaling for already-ropetuned models - just use their values
2023-11-24 14:20:36 +08:00
Concedo
dc4078c039
fixed segfault with all non-gguf models
2023-11-20 22:31:56 +08:00
Concedo
22c56f9221
default to multiuser
2023-11-18 12:55:59 +08:00
Concedo
a3f708afce
added more fields to the openai compatible completions APIs
2023-11-16 00:58:08 +08:00
Concedo
8b919b5b57
allow customized rope to use model set values
2023-11-15 16:21:52 +08:00
Concedo
f4ee91abbb
improved estimation
2023-11-13 15:45:13 +08:00
Concedo
be92cfa125
added preloadstory
2023-11-10 13:05:22 +08:00
Concedo
7ef4ec3b16
added trim_stop flag
2023-11-09 16:55:44 +08:00
Concedo
afa466807d
nooby layer selector considers contextsize
2023-11-09 14:05:35 +08:00
Concedo
fb3bcac368
handle memory separately for kcpp
2023-11-07 17:15:14 +08:00
Concedo
ea81eae189
cleanup, up ver (+1 squashed commits)
...
Squashed commits:
[1ea303d6] cleanup , up ver (+1 squashed commits)
Squashed commits:
[79f09b22] cleanup
2023-11-05 22:49:23 +08:00
YellowRoseCx
e2e5fe56a8
KCPP Fetches AMD ROCm Memory without a stick, CC_TURING Gets the Boot, koboldcpp_hipblas.dll Talks To The Hand, and hipBLAS Compiler Finds Its Independence! ( #517 )
...
* AMD ROCm memory fetching and max mem setting
* Update .gitignore with koboldcpp_hipblas.dll
* Update CMakeLists.txt remove CC_TURING for AMD
* separate hipBLAS compiler, update MMV_Y, move CXX/CC print
separate hipBLAS compiler, update MMV_Y value, move the section that prints CXX and CC compiler name
2023-11-05 22:23:18 +08:00
Concedo
5e5be717c3
fix for removing inaccessible backends in gui
2023-11-05 10:12:12 +08:00
Concedo
1e7088a80b
autopick cublas in gui if possible, better layer picking logic
2023-11-05 01:35:27 +08:00
Concedo
135001abc4
try to make the tunnel more reliable
2023-11-04 09:18:19 +08:00
Concedo
36f43ae834
syntax correction
2023-11-04 00:03:45 +08:00
Concedo
373c20ad51
print error log if tunnel fails
2023-11-03 23:48:21 +08:00
Concedo
879061c5d5
noavx2 clblast selector
2023-11-02 23:13:16 +08:00
Concedo
b0c7b88eac
try fix clouflare tunnel (+2 squashed commit)
...
Squashed commit:
[87d96bf2] update remote option
[c30bc909] updated fixed colab (+1 squashed commits)
Squashed commits:
[97b77563] updated fixed colab (+2 squashed commit)
Squashed commit:
[d851b04c] replaced cloudflare manual dl with remotetunnel in colab
[90ff1790] updated lite
2023-11-02 22:27:35 +08:00
Concedo
fca7a4c054
added noavx2 model for clblast (+1 squashed commits)
...
Squashed commits:
[291ecae6] added noavx2 mode for clblast (+1 squashed commits)
Squashed commits:
[562bc872] wip adding noavx2 cl
2023-11-02 15:22:34 +08:00
Concedo
82267e5e69
switched back to clinfo since it's possibly more cross platform and can get memory vals easily
2023-11-02 14:12:05 +08:00
Concedo
21588cefd4
tunnel code done (+1 squashed commits)
...
Squashed commits:
[b4bc7d20] wip integration of trycloudflare
2023-11-01 23:28:23 +08:00
Concedo
3b227fc704
automatic gpu layer detection
2023-11-01 20:55:26 +08:00
Concedo
b395dbf6f5
wip layer calculator
2023-11-01 20:04:10 +08:00
Concedo
ae2cd56de8
kobold integration of min_p sampler (+1 squashed commits)
...
Squashed commits:
[8ad2e349] kobold integration for min_p sampler
2023-11-01 19:08:45 +08:00
Concedo
df7e757d40
windows: added simpleclinfo, which helps determine clblast platform and device on windows
2023-11-01 18:10:35 +08:00
Concedo
f3690ba6d2
shifting enabled by default
2023-10-31 21:41:57 +08:00
Concedo
61c395833d
context shifting is still buggy
2023-10-30 16:25:01 +08:00
Concedo
7f5d1b2fc6
slider error
2023-10-30 00:02:38 +08:00
Concedo
7924592a83
context shift feature done
2023-10-29 18:21:39 +08:00
Concedo
09c74ea046
include content-length
2023-10-28 14:24:37 +08:00
Concedo
15f525c580
revamped smart context for llama models
2023-10-28 12:59:08 +08:00
Concedo
c2f675133d
support for abort without crash on disconnect
2023-10-27 15:27:17 +08:00
Concedo
aed05e5565
todo: troubleshoot sse with multiuser
2023-10-27 00:21:52 +08:00
Concedo
5db89b90b7
Merge branch 'master' into concedo_experimental
...
# Conflicts:
# .gitignore
# CMakeLists.txt
# Makefile
# README.md
# build.zig
# ggml-opencl.cpp
# tests/CMakeLists.txt
# tests/test-double-float.cpp
# tests/test-sampling.cpp
2023-10-25 23:58:15 +08:00
Concedo
98d1dba256
tighten timings
2023-10-25 20:44:20 +08:00
Concedo
cff75061fe
fixed some old models failing due to tokenizer changes, update lite (+1 squashed commits)
...
Squashed commits:
[9dee81ec] fixed some old models failing due to tokenizer changes, update lite tooltip (+3 squashed commit)
Squashed commit:
[5ab95a79] fixes
[a561d5e2] fixed some old models failing due to tokenizer changes
[95e65daf] lite updates
2023-10-22 11:04:59 +08:00
Concedo
6fa681b692
fixed a race condition with SSE streaming
2023-10-20 22:01:09 +08:00
Concedo
4382e51719
updated lite and default horde ctx amount
2023-10-19 22:49:59 +08:00
Concedo
6f8fe88f10
fix for lite (+5 squashed commit)
...
Squashed commit:
[f9ce9855] catch more exceptions
[8cdaf149] tweaked horde worker timeouts, updated lite
[619ebef4] fixed abort no response if failed
[a54a66a2] fixed time overflow
[9affdc3e] updated lite
2023-10-17 23:04:32 +08:00
Concedo
643902fbbb
fixed tensor split save and load
2023-10-13 10:07:22 +08:00
Concedo
7e2f714c9c
tensor split only for cuda
2023-10-12 17:01:52 +08:00
Alexander Abushady
11b8f97c1e
Tensor split UI ( #471 )
...
* update .gitignore
Remove .idea folder created by Jet Brains products.
* Front end, and partial backe-end
Tensor Split pulled in, shows in console, then not respected on model load.
* UI Tweak + Tensor Split Fix
Made Tensor Flow input match similar boxes around it. Also, fixed Tensor Split to populate the correct argument.
* Changed int to float for tensor split
Accidentally set int, needed to be float when setting tensor split args
2023-10-12 16:50:21 +08:00
Concedo
8be043ee38
more horde optimizations
2023-10-12 16:20:52 +08:00
Concedo
8d1cd512e2
missed a flag
2023-10-12 15:00:51 +08:00
Concedo
c6fe820357
improve cors and header handling
2023-10-12 14:53:39 +08:00
Concedo
f604cffdce
multiuser racer bugfix
2023-10-12 13:39:12 +08:00
