vrr/kvcache-ai-ktransformers
2
0
Fork 0
mirror of https://github.com/kvcache-ai/ktransformers.git synced 2025-09-09 13:55:27 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

modify moeinfer param

Browse source
This commit is contained in:
Azure 2025-02-06 14:07:38 +00:00
parent ee24a27001
commit 027b11266c
2 changed files with 3 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

2
ktransformers/local_chat.py
View file

@ -81,8 +81,10 @@ def local_chat():
content = "hi"
else:
content = open(config.prompt_file, "r").read()
print("User: ", content)
elif os.path.isfile(content):
content = open(content, "r").read()
print("User: ", content)
messages = his_content + [{"role": "user", "content": content}]
async def async_inference(messages):