vrr/koboldcpp
2
0
Fork 0
mirror of https://github.com/LostRuins/koboldcpp.git synced 2025-09-10 09:04:36 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
3447 commits 3 branches 196 tags 386 MiB
Commit graph

412 commits

Author SHA1 Message Date
Concedo
603fe941c1 increase cloudflared check size 2024-02-12 17:19:58 +08:00
Concedo
6f3196ad8e fix benchmark line 2024-02-10 21:49:14 +08:00
Concedo
c3d1a7d123 benchmark coherence fix 2024-02-09 19:03:48 +08:00
Concedo
35111ce01a row split mode is now a toggle 2024-02-09 18:35:58 +08:00
Concedo
d1aff0e964 benchmark only save under 1mb 2024-02-09 15:40:29 +08:00
Concedo
992eea71d7 fixes for vulkan multigpu 2024-02-09 14:42:27 +08:00
Concedo
fe424a5466 tensor split active text 2024-02-09 12:02:23 +08:00
Concedo
4cd571db89 vulkan multigpu, show uptime 2024-02-08 16:54:38 +08:00
Concedo
de7be2f4e0 benchmarker done 2024-02-07 22:04:53 +08:00
Concedo
5cd9b1d23a placeholder for benchmark 2024-02-06 21:48:07 +08:00
Concedo
f43667f499 runmode untouched fix 2024-02-05 21:52:33 +08:00
Concedo
330921db15 runmode untouched fix 2024-02-05 20:26:08 +08:00
Alexander Abushady
4cb956c7db
Quadratic Sampling UI (#652) 
* Quadratic Sampling UI

Kalomaze's Quadratic Sampling, now has a UI within KCPP.

* remove debug prints

* cleanup, add smooth sampler to dynatemp

---------

Co-authored-by: Concedo <39025047+LostRuins@users.noreply.github.com>
 2024-02-04 16:26:27 +08:00
Concedo
d229150d28 Checkpoint to test for speed 2024-01-31 22:26:33 +08:00
Concedo
340fbbbb04 show warning if genamt >= ctxsize, show t/s values 2024-01-31 18:51:42 +08:00
Concedo
916780eaf4 fixed a bug with stop seq processing 2024-01-31 15:16:08 +08:00
Ira Peach
e00e17b3f9
Flush STDOUT when server starts listening. (#651) 
This works around a Win32 issue when piping output from a PyInstaller
context, such as when doing so in a perl script or to an output file.
Print statements from a Python context don't properly get output unless
flushed.

This strategically flushes the print statements so no information is
lost, though it may be better to flush all print statements in a Python
context via a subroutine wrapper.

See also:

    https://mail.python.org/pipermail/python-bugs-list/2004-August/024923.html
    https://stackoverflow.com/a/466849
    https://stackoverflow.com/q/62693079
 2024-01-31 14:40:45 +08:00
Concedo
f81404e33c updated class py, added imatrix 2024-01-28 22:37:11 +08:00
Concedo
c2e497ccfb deferred aborting for queued generations 2024-01-28 14:24:15 +08:00
Concedo
61ca3a0d30 show total of 8 backends 2024-01-27 17:05:33 +08:00
Concedo
87d852b85c get gpu names with vulkaninfo 2024-01-26 12:58:30 +08:00
Concedo
2a4a7241e6 Merge branch 'vulkan_test' into concedo_experimental 
# Conflicts:
#	CMakeLists.txt
#	Makefile
#	llama.cpp
 2024-01-25 23:01:44 +08:00
Concedo
346c1a97de fixed file select cancel, updated lite 2024-01-24 16:36:53 +08:00
Concedo
0f6fa6be93 try adding other fallback backends for linux 2024-01-23 23:37:56 +08:00
Concedo
a4ed5c6471 added 48k ctx option 2024-01-23 17:27:02 +08:00
Concedo
08236ccc97 better abort handling, added support for dynatemp exponent 2024-01-23 16:56:12 +08:00
Concedo
dc7bc0cb50 Merge commit '584d674be6' into concedo_experimental 
# Conflicts:
#	.github/workflows/nix-flake-update.yml
#	Makefile
#	Package.swift
#	ggml-cuda.cu
#	tests/test-quantize-fns.cpp
 2024-01-14 16:29:44 +08:00
kalomaze
bd77a48037
Do not default to Repetition Penalty 1.1 (#615) 
* Do not default to Repetition Penalty

* apply all known aliases for repetition penalty when using the OAI endpoint. rep pen defaults to 1, range to 256

---------

Co-authored-by: Concedo <39025047+LostRuins@users.noreply.github.com>
 2024-01-13 22:20:02 +08:00
Concedo
b9ad08af19 improved dynatemp wizard 2024-01-11 11:26:14 +08:00
Concedo
5cc64ebb52 dynatemp wizard 2024-01-09 15:51:32 +08:00
Concedo
550829ed98 dont get stuck if cloudflared failed to download correctly 2024-01-08 21:11:17 +08:00
kalomaze
123bff9a0f
Full DynaTemp implementation + UI (#600) 
* move Dynatemp changes to new branch

* fix float header

* Properly reintroduce variable expert count

Controllable through experts.txt

* first pass at DynaTemp UI

Checkbox partial implemented, Min and Max Temp implemented

* DynaTemp UI Checkbox

Trigger DynaTemp on checkbox

* DynaTemp UI checkbox edition

Hell Yeah! DynaTemp!

* Remove greedy dynatemp

* Fix race condition caused by debug print

* Fixed broken presets and miro

Fixes broken presets and mirostat

* Remove debug function + HHI temp

Also removed unnecessary softmax double precision

* Fix whitespace (?) for generate function

* epic upstream renaming scheme fix

* fix stupid indents

* Other cleanup

Reintroduce unused rep pen function, move temp functions first before entropy dynamic temp

* Slight indent fix

* revert batch pyinstaller maker to mainline

and also delete experts.txt since adjustable routing is also being removed for the PR

* compact dynatemp into a single value dynatemp_range. This is a float which represents the allowed deviation from the min and max temperature when using dynatemp. Thus, if we want a value of dynatemp_min=0.3, dynatemp_max=0.5, then we would simply set temperature=0.4 and dynatemp_range=0.1. Functionally dynatemp would operate the same, but it would simplify usage and make it a single easy to adjust value.

---------

Co-authored-by: Alexander Abushady <aabushady214@gmail.com>
Co-authored-by: Concedo <39025047+LostRuins@users.noreply.github.com>
 2024-01-06 11:13:16 +08:00
Concedo
427ba21e62 add stub values for usage, revert cuda malloc pool implementation (+1 squashed commits) 
Squashed commits:

[fd4cfb44] add stub values for usage, revert cuda malloc pool implementation
 2024-01-05 21:58:16 +08:00
Concedo
20261049c9 try to reuse cloudflared file 2024-01-05 18:04:09 +08:00
Concedo
234f79fe9d Merge branch 'master' into concedo_experimental 
# Conflicts:
#	CMakeLists.txt
#	ci/run.sh
#	llama.cpp
 2024-01-03 22:33:38 +08:00
Concedo
94e68fe474 added field to show recent seed 2024-01-02 15:35:04 +08:00
Concedo
eee674045e use native cl if found 2023-12-31 00:53:22 +08:00
Concedo
6177196052 tweak tooltips 2023-12-30 11:02:30 +08:00
Concedo
7ad92dbf4a cleaned up the quick tab based on the suggested removals from discord members. 2023-12-30 10:41:46 +08:00
Concedo
63b65efb78 added tooltips for all items in the GUI launcher 2023-12-28 23:08:57 +08:00
Concedo
ec46661a32 wip adding tooltips 2023-12-28 15:54:22 +08:00
DebuggingLife46
e733a9e425
Add logit_bias to the OpenAI api (#577) 
* Add logit_bias to the OpenAI api

* Cleanup and refactor, test in swagger.

---------

Co-authored-by: Concedo <39025047+LostRuins@users.noreply.github.com>
 2023-12-27 00:26:19 +08:00
Concedo
c2d87b6545 increase multiuser default 2023-12-25 23:49:45 +08:00
Concedo
78a9d206d3 randomize horde genkey 2023-12-25 22:47:21 +08:00
Concedo
cc64f2cad1 Merge branch 'master' into concedo_experimental 
# Conflicts:
#	.github/ISSUE_TEMPLATE/bug.md
#	Makefile
#	README.md
#	ggml-cuda.cu
#	tests/test-grad0.cpp
 2023-12-25 18:47:21 +08:00
Concedo
bd0d9039ec better approach to multiuser check 2023-12-24 20:03:33 +08:00
Concedo
bc24c9334c prevent prompt leakage during usage of check endpoint when genkey is provided in multiuser mode 2023-12-24 17:08:43 +08:00
Concedo
8823e8b06d added presence penalty into lite ui 2023-12-23 10:39:40 +08:00
Concedo
852ca780c9 cherrypicked the Hipblas fixed from PR #571 2023-12-22 21:29:20 +08:00
Concedo
77463e0e9c batch size improvements 2023-12-22 15:27:40 +08:00