16 commits

Author	SHA1	Message	Date
Concedo	ad5676810a	merge CLBlast improvements - GPU dequant	2023-04-16 01:17:40 +08:00
Concedo	8ad42a1102	read from inputs	2023-04-14 21:30:26 +08:00
Concedo	adb4df78d6	Added SmartContext mode, a way of prompt context manipulation that avoids frequent context recalculation.	2023-04-14 21:24:16 +08:00
Concedo	5c22f7e4c4	reduce batch sizes and skip all intrinsic flags except AVX when building in compatibility mode.	2023-04-13 11:32:05 +08:00
Concedo	1bd5992da4	clean and refactor handling of flags	2023-04-12 23:25:31 +08:00
Concedo	23c675b2e6	integrated optional (experimentl) CLBlast support	2023-04-11 23:33:44 +08:00
0cc4m	c3db99ea32	Allow use of OpenCL GPU-based BLAS using ClBlast instead of OpenBLAS for context processing	2023-04-10 18:20:40 +02:00
Concedo	f53238f570	Merged the upstream updates for model loading code, and ditched the legacy llama loaders since they were no longer needed.	2023-04-10 12:00:34 +08:00
Concedo	b91abc3316	increase default blas batch size	2023-04-09 15:27:43 +08:00
Concedo	3d650d0e25	remove dependency of psutil, fixed compile error on WSL, handle exceptions when sending http response, added multiline for embedded kobold	2023-04-06 11:08:19 +08:00
Concedo	52de932842	removed main.exe to reduce clutter, added support for rep pen in gptj	2023-04-04 20:43:13 +08:00
Concedo	8dd8ab1659	Various enhancement and integration pygmalion.cpp	2023-04-03 00:04:43 +08:00
Concedo	9aabb0d9db	massive refactor completed, GPT-J integrated	2023-04-02 17:03:30 +08:00
Concedo	085a9f90a7	still refactoring	2023-04-01 11:56:34 +08:00
Concedo	801b178f2a	still refactoring, but need a checkpoint to prepare build for 1.0.7	2023-04-01 08:55:14 +08:00
Concedo	6b86f5ea22	halfway refactoring, wip adding other model types	2023-04-01 01:13:05 +08:00