vrr/kvcache-ai-ktransformers
2
0
Fork 0
mirror of https://github.com/kvcache-ai/ktransformers.git synced 2025-09-06 12:40:02 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 2

Merge branch 'feat-DeepSeekV3' of github.com:kvcache-ai/ktransformers into feat-DeepSeekV3

Browse source
This commit is contained in:
liam 2025-02-06 22:39:48 +08:00
commit 0262f954c7
1 changed files with 2 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
ktransformers/local_chat.py
View file

@ -81,8 +81,10 @@ def local_chat():
content = "hi"
else:
content = open(config.prompt_file, "r").read()
print("User: ", content)
elif os.path.isfile(content):
content = open(content, "r").read()
print("User: ", content)
messages = his_content + [{"role": "user", "content": content}]
async def async_inference(messages):