talemate/docs/user-guide/agents/visualizer/comfyui.md
veguAI 95a17197ba
0.26.0 (#133)
* implement manually disabling and enabling clients

* relock

* fix warning spam

* start moving stuff around

* move more stuff

* start separating world state manager into more managable submodules

* character title

* scroll home to top always

* finish separating character state editor into components

* fix defered nav to character sections

* separate components for pin and contextdb managing

* fix issue with context character filter search

* fix world state manage ux state reset issues

* wsm menu refactor
allow updating character image from wsm
cover image layout fixes

* remove debug spam

* fix client deletion / disabling rubber banding issue

* deactivate / activate / delete characters through wsm

* reload character instead

* fix koboldcpp client jiggle arguments

* save scene title

* fix deferred nav

* fix issue where blanking a character detail would bug out

* some layout changes

* character import copies cover image

* remove debug message

* character import via wsm

* deactivate imported characters

* images nav option placeholder

* start move towards new world state templating system

* prompt tweak

* add templates/world-state/*.yaml

* switch to new world state template system in manager

* template editor progress

* more wsm template changes

* template applicator component

* template applicate added to attributes and details

* selective template application

* fix issue with template editing

* attribute and detail templates dont require instructions

* adjust character attributes and details template applicator integration

* add gpt-4o

add gpt-4o-2024-05-13

* autocomplete prompt and postprocessing tweaks

* prompt tweaks

* fix issue where saving a new scene could cause recent config changes to revert

* only download punkt if its not downloaded yet

* working character attribute templates

* character detail generate working
move template generate logic to worldstate.templates

* character creator first steps

* support contextual generate when character doesn't exist

* move talemate wsm templates to their own dir, add supports_spice and supports_style flags

* wsm character creator progress

* character creator progress

* character creator progress and wire up image creation in character editor

* templating progress

* contextual generate generation options

* ux tweaks

* wirte up writing style and spice to generation

* wire spice / writing style to detail generation

* notify when spice is applied

* tweaks to generation spice notifications

* add some help / information to template editor

* fix some issues with detail and attribute generation

* some context gen tweaks

* character gen tweaks

* character color changer

* link to templates form gen option ux

* gen options for dialogue example genrate

* ctrl click to max spice level

* unify spice application notification into a component for reuse

* improvements to example dialogue generation

* some refinements to character editor

* remove some old cruft from scene schema

* wsm scene editor progress

* relock

* relock

* debug message cleanup

* fix issue with tab selection when loading a scene

* scene editor progress

* centralized generation options

* pass generation settings through to character creator

* save changes from wsm view

* scene settings
save copy

* refactor world entry / states editor

* fix issue with applying non-character world state templates

* layout fixes

* allow updating of scene cover image

* move history manager to world editor

* add phi-3 base template

* dialogue cleanup improvements

* refactor scoped game-engine api

* separate legacy creator functions to own file

* remove cruft

* some cleanup and fixes

* add photo style

* remove noisy log message

* better handling of active scene

* some fixes to pin editor

* don't enforce height

* active scene context fixes

* fix intro and scene description generration

* tweak preset for scene direction and summarization tasks

* ensure memory db is open

* update frontend dependencies

* update frontend dependencies

* fix issue with prompt query_memory function returning None

* typo

* default  world state templates

* new scene creation fixes
remove legacy creator ux

* scene export

* fix scene loading from upload

* add claude 3.5 sonnet

* fix automatic client selection when the current client is disabled

* remove cruft

* agent modal extended to support multiple config panels
visual agent prompt prefixes and suffixes addeed

* fix issue with world state template group saving

* resolve attribute name issue `copy`

* RequestInput: fix form validation and keystroke submit

* support chara load from json files also refactor character loading to load.py

* implement simple act-as feature using tab to cycle through active characters in the scene

* docs progress

* tts settings tweaks

* fix issue with loading older talemate scenes

* docs progress

* fix issue with config validation on new installs

* some tweaks for agent setting modals

* default template changed to alpaca

* docs dependencies

* gemma2 template

* nemotron4 template

* docs

* docs

* docs

* change prompt template section to autocomplete

* fix agent config not loading for some agents

* allow deletion of player character

* fix some oddities with scene outline commit

* automatically active player characters and create player characters with the correct actor class

* also set the first npc created as immediately acitve

* add has_active_npcs property and re-emit message history when scene outline is updated.

* indicate when visualizer is busy in the scene tools

* check for busy instead

* prompt tweaks for movie script type dialogue format

* gemma2 prompt fixed

* scene message colors updated

* act as narrator

* move to _old

* scene message appearance tweaks

* fix rubberbanding when editing text field in agent configs

* fix autocompletionm when acting as different character or narrator

* disable autocomplete during command execution

* remove autocomplete button from scene tools

* docs

* relock

* docs

* docs

* improve context pins in dialogue context

* better approximate token count

* fix pin condition editing

* fix issue where scene save as would lose long term memory entries

* immediately clean message history when loading a new scene

* docs

* ensure intro text has formatting markers

* narrator messages written by the player can now be deleted.

* scene editor

* move docs around

* start character editor docs

* more character editor docs:

* fix some ux bugs

* fix template group deletrion not removing the file

* docs

* typos

* docs

* relock

* docs

* notify image generation errors

* linting

* gh pages workflow

* use poetry

* dont use poetry

* link to docs site

* set site_url

* add trailing slash

* fix image paths

* re-add tabbyai link

* fix image generation error triggering incorrectly

* fix intro formatting incosistencies

* remove cruft

* add time passed label to history view

* date adjustments

* tests

* add gpt-4o-mini

* fix links

* remove hard ntlk requirement for voice generation chunking

ntlk error handling

fix typo

* docs

* fix issdue with dupe character card intro text

* disable character forms while templates are being applied.

* failure during context generate no longer locks ux

* refactor client and agent status display in system bar

* llama 3.1 8b claude

* fix format

* adjustments to automcomplete dialogue instructions

* add mistral nemo

* debug info

* fix system agent status getting stuck

* readme

* readme

* fix autocomplete responses when they are framed by quotes
2024-07-26 21:43:06 +03:00

2 KiB

ComfyUI

!!! info This requires you to setup a local instance of the ComfyUI API. Follow the instructions from their GitHub to get it running.

Once you're setup, copy their start.bat file to a new start-listen.bat file and change the contents to.

call venv\Scripts\activate
call python main.py --port 8188 --listen 0.0.0.0

Then run the start-listen.bat to start the API.

Once your ComfyUI API is running (check with your browser) you can set the Visualizer config to use the ComfyUI backend.

Settings

Visual agent comfyui settings

API URL

The url of the API, if following this example, should be http://localhost:8188

Workflow

The workflow file to use. This is a comfyui api workflow file that needs to exist in ./templates/comfyui-workflows inside the talemate directory. Talemate provides two very barebones workflows with default-sdxl.json and default-sd15.json. You can create your own workflows and place them in this directory to use them.

!!! warning The workflow file must be generated using the API Workflow export, not the UI export. Please refer to their documentation for more information.

Checkpoint

The model to use - this will load a list of all available models in your comfyui instance. Select which one you want to use for the image generation.

!!! tip If your list of models is empty, try disabling and re-enabling the visualizer agent. This will force a refresh of the models list.

Custom Workflows

When creating custom workflows for ideal compatibility with Talemate, ensure the following.

  • A CheckpointLoaderSimple node named Talemate Load Checkpoint
  • A EmptyLatentImage node name Talemate Resolution
  • A ClipTextEncode node named Talemate Positive Prompt
  • A ClipTextEncode node named Talemate Negative Prompt
  • A SaveImage node at the end of the workflow.

ComfyUI Base workflow example