Talemate

Allows you to play roleplay scenarios with large language models.

It does not run any large language models itself but relies on existing APIs. Currently supports text-generation-webui and openai.

This means you need to either have an openai api key or know how to setup oobabooga/text-generation-webui (locally or remotely via gpu renting.)

Current features

• responive modern ui
• agents
  • conversation
  • narration
  • summarization
  • director
  • creative
• multi-client (agents can be connected to separate LLMs)
• long term memory (very experimental at this point)
• narrative world state
• narrative tools
• creative tools
  • AI backed character creation with template support (jinja2)
  • AI backed scenario creation
• runpod integration
• overridable templates foe all LLM prompts. (jinja2)

Planned features

Kinda making it up as i go along, but i want to lean more into gameplay through AI, keeping track of gamestates, moving away from simply roleplaying towards a more game-ified experience.

In no particular order:

• Gameplay loop governed by AI
• Improved world state
• Dynamic player choice generation
• Better creative tools
  • node based scenario / character creation
• Improved long term memory (base is there, but its very rough at the moment)
• Improved director agent
  • Right now this doesn't really work well on anything but GPT-4 (and even there it's debatable). It tends to steer the story in a way that introduces pacing issues. It needs a model that is creative but also reasons really well i think.
• Automatic1111 client

Quickstart

Installation

Windows

1. Download and install Python 3.10 or higher from the official Python website.
2. Download and install Node.js from the official Node.js website. This will also install npm.
3. Download the Talemate project to your local machine. Download from the Releases page.
4. Unpack the download and run install.bat by double clicking it. This will set up the project on your local machine.
5. Once the installation is complete, you can start the backend and frontend servers by running start.bat.
6. Navigate your browser to http://localhost:8080

Linux

python 3.10 or higher is required.

1. git clone git@github.com:final-wombat/talemate
2. cd talemate
3. source install.sh
4. Start the backend: python src/talemate/server/run.py runserver --host 0.0.0.0 --port 5001.
5. Open a new terminal, navigate to the talemate_frontend directory, and start the frontend server by running npm run serve.

Configuration

OpenAI

To set your openai api key, open config.yaml in any text editor and uncomment / add

openai:
    api_key: sk-my-api-key-goes-here

You will need to restart the backend for this change to take effect.

RunPod

To set your runpod api key, open config.yaml in any text editor and uncomment / add

runpod:
    api_key: my-api-key-goes-here

You will need to restart the backend for this change to take effect.

Once the api key is set Pods loaded from text-generation-webui templates (or the bloke's runpod llm template) will be autoamtically added to your client list in talemate.

ATTENTION: Talemate is not a suitable for way for you to determine whether your pod is currently running or not. Always check the runpod dashboard to see if your pod is running or not.

Recommended Models

Note: this is my personal opinion while using talemate. If you find a model that works better for you, let me know about it.

Will be updated as i test more models and over time.

Model Name	Type	Notes
GPT-4	Remote	Still the best for consistency and reasoning, but is heavily censored, while talemate will send a general "decensor" system prompt, depending on the type of content you want to roleplay, there is a chance your key will be banned. If you do use this make sure to monitor your api usage, talemate tends to send a lot more requests than other roleplaying applications.
Nous Hermes LLama2	13B model	My go-to model for 13B parameters. It's good at roleplay and also smart enough to handle the world state and narrative tools. A 13B model loaded via exllama also allows you run chromadb with the xl instructor embeddings off of a single 4090.
MythoMax	13B model	Similar quality to Hermes LLama2, but a bit more creative. Rarely fails on JSON responses.
Synthia v1.2 34B	34B model	Cannot be run at full context together with chromadb instructor models on a single 4090. But a great choice if you're running chromadb with the default embeddings (or on cpu).
Genz	70B model	Great choice if you have the hardware to run it (or can rent it).
Synthia v1.2 70B	70B model	Great choice if you have the hardware to run it (or can rent it).

I have not included OpenAI's gpt-3.5-turbo in this list, since it is really inconsistent with JSON responses, plus its probably still just as heavily censored as GPT-4.

I have not tested with Llama 1 models in a while, Lazarus was really good at roleplay, but started failing on JSON requirements.

I have not tested with anything below 13B parameters.

Connecting to an LLM

On the right hand side click the "Add Client" button. If there is no button, you may need to toggle the client options by clicking this button:

Text-generation-webui

In the modal if you're planning to connect to text-generation-webui, you can likely leave everything as is and just click Save.

OpenAI

If you want to add an OpenAI client, just change the client type and select the apropriate model.

Ready to go

You will know you are good to go when the client and all the agents have a green dot next to them.

Load the introductory scenario "Infinity Quest"

Generated using talemate creative tools, mostly used for testing / demoing.

You can load it (and any other talemate scenarios or save files) by expanding the "Load" menu in the top left corner and selecting the middle tab. Then simple search for a partial name of the scenario you want to load and click on the result.

Loading character cards

Supports both v1 and v2 chara specs.

Expand the "Load" menu in the top left corner and either click on "Upload a character card" or simply drag and drop a character card file into the same area.

Once a character is uploaded, talemate may actually take a moment because it needs to convert it to a talemate format and will also run additional LLM prompts to generate character attributes and world state.

Make sure you save the scene after the character is loaded as it can then be loaded as normal talemate scenario in the future.

Further documentation

• Creative mode (docs WIP)
• Prompt template overrides
• ChromaDB (long term memory)
• Runpod Integration