vrr/chat-relay

Fork 0

mirror of https://github.com/BinaryBeastMaster/chat-relay.git synced 2026-04-28 03:39:27 +00:00

BinaryBeastMaster 25abcd2974 Initial commit

2025-05-10 15:09:23 -07:00

33 KiB

Raw Permalink Blame History

Chat Relay System: Comprehensive User Manual

Introduction
System Architecture
Installation and Setup
Configuration
Usage Guide
MCP Server Functionality
Troubleshooting
Developer Reference
Appendices

Introduction

Purpose

The Chat Relay System enables communication between AI assistants like Cline/RooCode and web-based AI chat interfaces such as Gemini, AI Studio, ChatGPT, and Claude. It provides an OpenAI-compatible API that allows these assistants to interact with web-based chat interfaces that may not have public APIs or have different capabilities through their web interfaces.

System Overview

The Chat Relay System consists of three main components:

API Relay Server: An Express.js server that implements an OpenAI-compatible API endpoint and manages WebSocket connections with browser extensions.
Browser Extension: A Chrome extension that connects to the API Relay Server via WebSocket and interacts with web-based chat interfaces.
MCP Server: A Model Context Protocol (MCP) server that provides additional tools and resources for the system, enabling AI assistants to send messages through the relay.

Together, these components create a seamless bridge between AI assistants and web-based chat interfaces, allowing for powerful interactions and capabilities.

Key Features

OpenAI-Compatible API: Implements the /v1/chat/completions endpoint that Cline/RooCode can connect to.
Real-time Communication: Uses WebSockets for efficient, real-time communication between components.
Multiple Chat Interface Support: Works with Gemini, AI Studio, ChatGPT, and Claude.
Modular Provider Architecture: Easily extendable to support additional chat interfaces.
MCP Integration: Provides tools for AI assistants to interact with the system programmatically.
Robust Error Handling: Includes timeout handling, retry mechanisms, and detailed logging.

System Architecture

Components

1. API Relay Server

The API Relay Server is an Express.js application that serves as the central hub of the Chat Relay System. It provides:

An OpenAI-compatible REST API for AI assistants to connect to
A WebSocket server for real-time communication with browser extensions
Request tracking and management
Response formatting and delivery

2. Browser Extension

The Browser Extension is a Chrome extension that:

Connects to the API Relay Server via WebSocket
Injects content scripts into supported chat interfaces
Sends messages to chat interfaces
Captures responses using DOM manipulation and the Chrome debugger API
Relays responses back to the API Relay Server

3. MCP Server

The MCP (Model Context Protocol) Server is a Node.js application that:

Provides tools for AI assistants to interact with the Chat Relay System
Enables sending messages through the API Relay Server
Receives responses from chat interfaces
Offers a standardized interface for AI assistants to use

Data Flow

The data flow through the Chat Relay System follows these steps:

AI Assistant to API Relay Server:
- AI assistant sends an HTTP POST request to /v1/chat/completions
- Request includes the message and optional parameters
API Relay Server to Browser Extension:
- API Relay Server assigns a unique requestId to the message
- Message is sent to the Browser Extension via WebSocket
Browser Extension to Chat Interface:
- Browser Extension identifies the appropriate chat interface
- Message is inserted into the chat input field
- Send button is clicked to submit the message
Chat Interface to Browser Extension:
- Browser Extension captures the response from the chat interface
- Response is associated with the original requestId
Browser Extension to API Relay Server:
- Response is sent back to the API Relay Server via WebSocket
- API Relay Server matches the response to the original request
API Relay Server to AI Assistant:
- Response is formatted in OpenAI-compatible JSON structure
- Formatted response is sent back to the AI assistant

sequenceDiagram
    participant A as AI Assistant
    participant R as API Relay Server
    participant E as Browser Extension
    participant C as Chat Interface
    
    A->>R: HTTP POST /v1/chat/completions
    R->>E: WebSocket: SEND_CHAT_MESSAGE (requestId)
    E->>C: Insert text & click send
    C->>E: Generate response
    E->>R: WebSocket: CHAT_RESPONSE_CHUNK (requestId)
    R->>A: HTTP Response (OpenAI format)

Port Configuration

The Chat Relay System uses several ports for communication between components:

Component	Default Port	Protocol	Description
API Relay Server	3003	HTTP	REST API endpoint for AI assistants
API Relay Server	3003	WebSocket	WebSocket server for browser extensions
MCP Server	N/A	stdio	Uses standard input/output for communication

Installation and Setup

Prerequisites

Before installing the Chat Relay System, ensure you have the following:

Node.js: Version 18.19.0 or higher
npm: Version 6.0.0 or higher
Chrome: Latest version recommended
Git: For cloning the repository

API Relay Server Setup

Clone the repository (if you haven't already):

git clone https://github.com/binarybeastmaster/chat-relay.git
cd chat-relay

Install dependencies:
```
cd api-relay-server
npm install
```

Start the server:

nodemon start

The server will start on port 3003 by default. You should see output similar to:

OpenAI-compatible relay server running on port 3003
WebSocket server for browser extensions running on ws://localhost:3003

Browser Extension Setup

Build the extension (if necessary):

cd extension
# No build step required for this extension

Install in Chrome:
- Open Chrome and navigate to chrome://extensions/
- Enable "Developer mode" using the toggle in the top-right corner
- Click "Load unpacked"
- Select the extension directory from your chat-relay project
Verify installation:
- The Chat Relay extension icon should appear in your Chrome toolbar
- Click the icon to see the extension popup
- The popup should indicate whether the extension is connected to the API Relay Server

MCP Server Setup

Install dependencies:
```
cd mcp-server
npm install
```
Build and install the server:
```
 npm run build
 npm install -g .
```

or package installation and install

npm run build
npm pack
npm install -g C:/Users/user/Projects/chat-relay/mcp-server/chat-relay-mcp-0.0.1.tgz


3. **Start the server**:
```bash
npm start

The MCP Server will start and be ready to receive commands. You should see output similar to:

Initializing Chat Relay MCP Server...
Registered tools using 'registerTool' method
MCP Server started with stdio transport. Registered operations: send_message
MCP Server is configured to communicate with API Relay Server on http://localhost:3003

Global Installation (optional): To make the MCP Server available globally, you can install it as a global npm package:
```
npm install -g ./
```
After global installation, you can start the MCP Server from anywhere using:
```
chat-relay-mcp
```

Configuration

API Relay Server Configuration

The API Relay Server is primarily configured via the Admin UI or by directly editing the api-relay-server/dist/server-config.json file (which is created and managed by the server). Some initial defaults can also be influenced by environment variables.

`server-config.json`

This JSON file is automatically created by the server in the api-relay-server/dist/ directory if it doesn't exist. It stores the following settings:

port: The port number the server listens on.
requestTimeoutMs: The timeout in milliseconds for waiting for a response from the browser extension.
newRequestBehavior: Determines how new requests are handled if the browser extension is already processing a request.
- "queue": New requests are queued and processed sequentially (default).
- "drop": New requests are rejected with a 429 "Too Many Requests" error.
lastRestartRequestTimestamp: Timestamp of the last requested server restart via the admin UI.

Example server-config.json:

{
  "port": 3003,
  "requestTimeoutMs": 120000,
  "newRequestBehavior": "queue",
  "lastRestartRequestTimestamp": 1678886400000
}

Admin UI Configuration

The Admin UI, accessible at http://localhost:<PORT>/admin (e.g., http://localhost:3003/admin), provides a "Settings" tab where you can configure:

Server Port:
- Displays the current port the server is configured to use.
- Changing this value requires a server restart to take effect. The new value is saved to server-config.json.
Request Timeout (ms):
- Displays and allows modification of the timeout for requests sent to the browser extension.
- This change is effective immediately and is saved to server-config.json.
New Request Behavior (if extension busy):
- Allows selection between 'Queue' and 'Drop' using radio buttons.
  - Queue: If the browser extension is busy, incoming requests will be held in a queue and processed one by one as the extension becomes free.
  - Drop: If the browser extension is busy, incoming requests will be immediately rejected with an error (429 Too Many Requests).
- This change is effective immediately and is saved to server-config.json.

Changes made in the Admin UI are saved to server-config.json.

Environment Variables

PORT: Can be used to set the initial port if server-config.json does not exist or does not contain a port setting. Example: PORT=3004 npm start.
REQUEST_TIMEOUT_MS: Can be used to set the initial request timeout if server-config.json does not exist or does not contain this setting.

The values from server-config.json take precedence over environment variables once the file is created.

Browser Extension Configuration

The Browser Extension can be configured through its options page or by editing the source code:

Options Page

Right-click the extension icon in Chrome
Select "Options"
Configure the following settings:
- Server Host: The hostname of the API Relay Server (default: localhost)
- Server Port: The port of the API Relay Server (default: 3003)
- Server Protocol: The WebSocket protocol to use (default: ws)

Source Code Configuration

You can also configure the Browser Extension by editing the extension/background.js file:

// Default settings
const DEFAULT_SETTINGS = {
  serverHost: 'localhost',
  serverPort: 3003,
  serverProtocol: 'ws'
};

// Reconnection settings
let reconnectInterval = 5000; // 5 seconds

MCP Server Configuration

The MCP Server configuration is defined in the mcp-server/src/index.ts file:

// Create the MCP server with configuration
const mcpServer = new MCPServer({
  transport: { type: "stdio" }
} as any);

The MCP Server is configured to communicate with the API Relay Server at http://localhost:3003. If you need to change this URL, edit the mcp-server/src/tools/SendMessageTool.ts file:

// Send a POST request to the API relay server
const response = await fetch('http://localhost:3003/v1/chat/completions', {
  // ...
});

Cline/RooCode Configuration

To configure Cline/RooCode to use the Chat Relay System:

Open Cline/RooCode settings
Set the API Provider to "OpenAI Compatible"
Configure the following settings:
- Base URL: http://localhost:3003
- API Key: Any value (not validated)
- Model ID: gemini-pro, chatgpt, or any label

Usage Guide

Basic Usage

Starting the System

Start the API Relay Server:
```
cd api-relay-server
npm start
```
Ensure the Browser Extension is installed in Chrome
Open a supported chat interface in Chrome:
- Gemini: https://gemini.google.com/
- AI Studio: https://aistudio.google.com/
- ChatGPT: https://chatgpt.com/
- Claude: https://claude.ai/
You should see a mesaage at the top of the page that says "AI chat relay" started debugging this browser.
Start the MCP Server (if needed):
```
cd mcp-server
npm start
```

Testing the System

You can test the Chat Relay System using curl:

curl -X POST http://localhost:3003/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "chatgpt",
    "messages": [{ "role": "user", "content": "Hello, tell me about mars" }],
    "temperature": 0.7,
    "max_tokens": 100
  }'

curl -X POST http://localhost:3003/v1/chat/completions -H "Content-Type: application/json" -d ' {"model":"chatgpt","messages":[{"role":"user","content":"Hello, tell me about mars"}],"temperature":0.7,"max_tokens":100}'

If everything is working correctly, you should see:

The message "Hello!" appear in the chat interface
The chat interface generate a response
The response returned in the curl output in OpenAI-compatible format

Note: If you dont see your message injected into the chat interface, you may need to refresh the page of your AI chat provider. If that doesnt work, first reload the extension they refresh the the page of your API chat provider and try again. Ensure that you see a mesaage at the top of the page that says "AI chat relay" started debugging this browser.

Using with Cline/RooCode

After configuring Cline/RooCode as described in the Configuration section:

Start the API Relay Server and ensure the Browser Extension is installed
Open a supported chat interface in Chrome
Add an OpenAI compatible provider. Enter http://localhost:3003/v1 as the base URL. Enter anything intot he authentication key as its currently unused but must have a value. I suggest to add a 3-5 second delay been messages in the provider settings as well as limited file reads to 500 lines or less.
Use Cline/RooCode normally - your messages will be relayed to the chat interface, and responses will be returned to Cline/RooCode

Using with MCP Server

The MCP Server provides a tool called send_message that allows AI assistants to send messages through the Chat Relay System:

// Example of using the send_message tool
const result = await use_mcp_tool({
  server_name: "chat-relay-mcp",
  tool_name: "send_message",
  arguments: {
    message: "Hello! This is a test message sent through the MCP server. How are you today?"
  }
});

The response will be returned as a string in the result variable.

Advanced Usage/Test Examples

Generating Code

You can use the Chat Relay System to generate code by sending appropriate prompts:

// Example of generating a Fibonacci function
const result = await use_mcp_tool({
  server_name: "chat-relay-mcp",
  tool_name: "send_message",
  arguments: {
    message: "Please write a JavaScript function that calculates the Fibonacci sequence up to n terms. Return only the code without any explanation or markdown formatting."
  }
});

Creating a React Component

// Example of creating a React component
const result = await use_mcp_tool({
  server_name: "chat-relay-mcp",
  tool_name: "send_message",
  arguments: {
    message: "Please write a simple React component that displays a counter with increment and decrement buttons. Return only the code without any explanation or markdown formatting."
  }
});

Adding Content to an HTML File

// Example of adding a contact form to an HTML file
const result = await use_mcp_tool({
  server_name: "chat-relay-mcp",
  tool_name: "send_message",
  arguments: {
    message: `I have the following HTML file:
<!DOCTYPE html>
<html>
<head>
  <title>My Website</title>
  <style>
    body { font-family: Arial, sans-serif; }
  </style>
</head>
<body>
  <h1>Welcome to My Website</h1>
  <!-- Content will be added here -->
</body>
</html>

Please add a contact form to this HTML file where the comment '<!-- Content will be added here -->' is. The form should include fields for name, email, subject, and message, along with a submit button. Also add appropriate CSS styles for the form. Return the complete modified HTML file.`
  }
});

Optimizing JavaScript Code

// Example of optimizing JavaScript code
const result = await use_mcp_tool({
  server_name: "chat-relay-mcp",
  tool_name: "send_message",
  arguments: {
    message: `I have the following JavaScript utility file:
// Utility functions
function isEven(num) {
  return num % 2 === 0;
}

function isOdd(num) {
  return num % 2 !== 0;
}

function calculateSum(arr) {
  let sum = 0;
  for (let i = 0; i < arr.length; i++) {
    sum += arr[i];
  }
  return sum;
}

function findMax(arr) {
  let max = arr[0];
  for (let i = 1; i < arr.length; i++) {
    if (arr[i] > max) {
      max = arr[i];
    }
  }
  return max;
}

function findMin(arr) {
  let min = arr[0];
  for (let i = 1; i < arr.length; i++) {
    if (arr[i] < min) {
      min = arr[i];
    }
  }
  return min;
}

module.exports = { isEven, isOdd, calculateSum, findMax, findMin };

Please optimize this file by:
1. Using modern JavaScript syntax (ES6+)
2. Adding input validation to each function
3. Optimizing the functions for performance where possible
4. Adding JSDoc comments for better documentation
Return the optimized version of the file.`
  }
});

MCP Server Functionality

Available Tools

The MCP Server provides the following tools:

send_message

Sends a message through the API Relay Server to the browser extension.

Input Schema:

{
  "message": "string"
}

Example Usage:

const result = await use_mcp_tool({
  server_name: "chat-relay-mcp",
  tool_name: "send_message",
  arguments: {
    message: "Hello from Claude! This is a test message sent through the MCP server."
  }
});

Sending Messages

The send_message tool sends a message to the API Relay Server, which then forwards it to the browser extension. The browser extension inserts the message into the chat interface and captures the response.

The message flow is as follows:

AI assistant calls the send_message tool with a message
MCP Server sends a POST request to the API Relay Server's /v1/chat/completions endpoint
API Relay Server forwards the message to the browser extension
Browser extension inserts the message into the chat interface
Chat interface generates a response
Browser extension captures the response and sends it back to the API Relay Server
API Relay Server returns the response to the MCP Server
MCP Server returns the response to the AI assistant

Integration with AI Models

The Chat Relay System allows AI assistants to interact with various AI models through their web interfaces:

Gemini: Google's conversational AI model
AI Studio: Google's AI development platform
ChatGPT: OpenAI's conversational AI model
Claude: Anthropic's AI model

By using the Chat Relay System, AI assistants can leverage the capabilities of these models even if they don't have direct API access.

Example Use Cases

1. Code Generation and Optimization

AI assistants can use the Chat Relay System to generate and optimize code by sending appropriate prompts to the chat interface. For example:

const result = await use_mcp_tool({
  server_name: "chat-relay-mcp",
  tool_name: "send_message",
  arguments: {
    message: "Please write a JavaScript function that calculates the average of an array of numbers. The function should follow the same style and validation patterns as the other functions in the utils.js file. It should use modern JavaScript syntax, include input validation, and have JSDoc comments."
  }
});

2. Content Creation

AI assistants can use the Chat Relay System to create content such as HTML, CSS, or text. For example:

const result = await use_mcp_tool({
  server_name: "chat-relay-mcp",
  tool_name: "send_message",
  arguments: {
    message: "Please write a blog post about the benefits of artificial intelligence in healthcare. The post should be around 500 words and include an introduction, three main points, and a conclusion."
  }
});

3. Problem Solving

AI assistants can use the Chat Relay System to solve complex problems by leveraging the capabilities of of AI models. This helfule as some AI models are good at using tools but may not be as "smart" as others. This mechanism gives you the best of both. For example:

const result = await use_mcp_tool({
  server_name: "chat-relay-mcp",
  tool_name: "send_message",
  arguments: {
    message: "I'm trying to optimize a database query that's running slowly. Here's the current query: SELECT * FROM users JOIN orders ON users.id = orders.user_id WHERE orders.status = 'pending'. Can you suggest ways to optimize this query?"
  }
});

Troubleshooting