vrr/koboldcpp
2
0
Fork 0
mirror of https://github.com/LostRuins/koboldcpp.git synced 2025-09-12 01:54:37 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 3
263 commits 3 branches 196 tags 389 MiB
Commit graph

20 commits

Author SHA1 Message Date
InconsolableCellist
f7c905b0d0 Minor overhaul of code: 
* Set number of utilized llama.cpp threads back to os.cpu_count, which
  had better performance on my machine (20 threads vs. 6, 3m12s vs.
  4m42s on 65B)

* Using argparse for command line args

* Supports binding to a specific interface, for use on LANs/WANs (no
  longer limited to just 127.0.0.1). Requires modified klite.embd

* General code cleanup and passing some parameters around without
  globals
 2023-03-28 13:39:34 -06:00
Concedo
46ddbb22bf allow url params 2023-03-27 17:40:05 +08:00
Concedo
57474944d6 Merge branch 'master' into concedo 
# Conflicts:
#	.github/workflows/build.yml
#	CMakeLists.txt
#	Makefile
#	README.md
 2023-03-26 14:52:08 +08:00
Concedo
119392f6f2 defaulting to f32 kv, and 4 threads seem to produce better results 2023-03-25 11:11:40 +08:00
Concedo
506cd62638 changed some defaults to hopefully increase compatibility 2023-03-25 10:40:11 +08:00
Concedo
b13a768813 added softprompt endpoint 2023-03-25 10:12:47 +08:00
Concedo
e791827973 added a GUI for selection of models if none was passed in through command line. 2023-03-24 22:03:57 +08:00
Concedo
c6c60332a4 Optimizations 2023-03-24 21:33:53 +08:00
Concedo
706e19e9b4 added ability to fast forward in time through partially duplicated prompts 2023-03-24 18:50:16 +08:00
Concedo
5f142df76e dynamic max context size defaulting to 1024, also implemented the basic API as a fallback 2023-03-22 15:56:47 +08:00
Henk
19178fa28e 2048 context all core 2023-03-21 21:49:47 +01:00
Concedo
7e6f4ed5e2 use a portable exit function 2023-03-21 22:00:27 +08:00
Concedo
a1625c4be1 changed file embedding technique 2023-03-21 21:16:06 +08:00
Concedo
ff1c0c4898 added embedded copy of kobold lite 2023-03-21 20:41:19 +08:00
Concedo
8d39365af6 update license, added backwards compatibility with both ggml model formats, fixed context length issues. 2023-03-20 23:43:35 +08:00
Concedo
dda69d4034 bugfixes 2023-03-20 13:37:51 +08:00
Concedo
29054a2bee explicit buffer allocation from python 2023-03-20 01:18:34 +08:00
Concedo
356c1b87ba bugfixes and support for persistent states 2023-03-20 00:59:45 +08:00
Concedo
f952b7c613 Removed junk, fixed some bugs and support dynamic number of sharded files 
Merge remote-tracking branch 'origin/master' into concedo

# Conflicts:
#	README.md
 2023-03-19 11:13:00 +08:00
Concedo
2c8f870f53 Created a python bindings for llama.cpp and emulated a simple Kobold HTTP API Endpoint 2023-03-19 00:07:11 +08:00