wangkuigang-yewu-cmss 4538bdae97 prevent rpc process from crashing on long prompt ... 当prompt超过cache_len的时候，rpc进程会crash掉，导致整体不可用。 这里增加一个检查，让过长的prompt在请求早期就被提前过滤掉		2025-04-13 16:13:16 +08:00
..
api	[update] support openai chat completion api	2025-03-07 08:51:09 +00:00
backend	prevent rpc process from crashing on long prompt	2025-04-13 16:13:16 +08:00
balance_serve	format kvc2, delete quant_configs, move model_configs to ~/.ktransformers	2025-04-08 10:06:07 +00:00
config	format kvc2, delete quant_configs, move model_configs to ~/.ktransformers	2025-04-08 10:06:07 +00:00
crud	Initial commit	2024-07-27 16:06:58 +08:00
models	Initial commit	2024-07-27 16:06:58 +08:00
schemas	add balance-serve, support concurrence	2025-03-31 22:55:32 +08:00
utils	add balance-serve, support concurrence	2025-03-31 22:55:32 +08:00
__init__.py	Initial commit	2024-07-27 16:06:58 +08:00
args.py	format kvc2, delete quant_configs, move model_configs to ~/.ktransformers	2025-04-08 10:06:07 +00:00
exceptions.py	Initial commit	2024-07-27 16:06:58 +08:00
main.py	add balance-serve, support concurrence	2025-03-31 22:55:32 +08:00
requirements.txt	add balance-serve, support concurrence	2025-03-31 22:55:32 +08:00