guanaco/README.md

Guanaco 🦙

Maximize your Ollama Cloud subscription.

Guanaco is a self-hosted FastAPI proxy that sits between your applications and Ollama Cloud. It provides an OpenAI-compatible /v1/chat/completions endpoint, emulates 8 major search and scrape APIs, tracks token usage, supports transparent fallback to external providers, and ships with a real-time management dashboard — all on a single port.

pip install guanaco

---

Features

• 🦙 LLM Router — OpenAI-compatible /v1/chat/completions and Anthropic-compatible /v1/messages proxy with streaming, token tracking, and analytics
• 🔍 8 Search/Scrape Emulators — Drop-in replacements for Tavily, Exa, SearXNG, Firecrawl, Serper, Jina, Cohere, and Brave Search
• 🔄 Fallback Provider — Automatically route to a secondary OpenAI-compatible provider when Ollama Cloud is slow, rate-limited, or unavailable
• 📊 Usage Tracking — Monitor Ollama Cloud session and weekly quota usage in real time
• 💾 Smart Caching — Optional exact-match and session-aware prefix caching (BETA) to reduce redundant API calls
• 📈 Web Dashboard — Real-time analytics, model configuration, API key management, and service status at http://localhost:8080/dashboard
• 🐳 Docker & systemd — Production-ready deployment with included service unit files
• 🔁 Backward Compatible — The oct CLI alias is fully preserved

---

Quick Start

1. Install

pip install guanaco

2. Configure

guanaco setup

Or set your API key directly:

export OLLAMA_API_KEY=your_ollama_api_key

3. Run

guanaco start

Your apps can now hit:

Endpoint	Purpose
http://localhost:8080/v1/chat/completions	OpenAI-compatible LLM router
http://localhost:8080/v1/messages	Anthropic-compatible proxy
http://localhost:8080/tavily/search	Tavily search (emulated)
http://localhost:8080/exa/search	Exa search (emulated)
http://localhost:8080/firecrawl/scrape	Firecrawl scrape (emulated)
http://localhost:8080/brave/search	Brave Search (emulated)
http://localhost:8080/dashboard	Web dashboard

---

CLI Commands

Command	Description
guanaco start	Start the proxy server (router + search APIs + dashboard)
guanaco setup	Interactive configuration wizard
guanaco status	Show service status and Ollama Cloud connectivity
guanaco models	List available Ollama Cloud models
guanaco models --refresh	Force-refresh model list from Ollama API
guanaco models --capabilities	Show model capabilities and sizes
guanaco usage	Check current Ollama Cloud session/weekly quota
guanaco key generate	Generate a new API key
guanaco key list	List all API keys
guanaco key revoke	Revoke an API key
guanaco analytics	View request analytics summary
guanaco analytics --errors	Show recent errors
guanaco analytics --model <name>	Show history for a specific model
guanaco config --show	Show current configuration
guanaco config --set <key> <value>	Update a config value
guanaco version	Show version

---

Dashboard

The built-in web dashboard is available at http://localhost:8080/dashboard.

┌────────────────────────────────────────────────────────────────┐
│  Dashboard Preview                                              │
│                                                                │
│  ┌──────────────┐  ┌──────────────┐  ┌──────────────────────┐│
│  │ Total Requests│  │ LLM Calls    │  │ Prompt Tokens        ││
│  │    1,249     │  │    892       │  │   4.2M               ││
│  └──────────────┘  └──────────────┘  └──────────────────────┘│
│                                                                │
│  ┌─────────────────────────────────────────────────────────┐  │
│  │ Usage: Session ████████░░░░░░░░░  62%  Resets in 12 min  │  │
│  └─────────────────────────────────────────────────────────┘  │
│                                                                │
│  [Models] [Analytics] [API Keys] [Config] [Status]           │
└────────────────────────────────────────────────────────────────┘

Features: real-time request analytics, token usage graphs, model configuration, fallback provider setup, API key management, and Ollama Cloud quota monitoring.

---

Configuration

Guanaco stores configuration in ~/.guanaco/config.yaml. You can change the config directory:

export GUANACO_CONFIG_DIR=/path/to/config

Full config.yaml Reference

# ── Required ──
ollama_api_key: "sk-ollama-..."       # Or set via OLLAMA_API_KEY env var

# ── Server ──
router:
  host: "127.0.0.1"                   # Bind address
  port: 8080                           # Listen port
  use_tailscale: false                # Use Tailscale IP for endpoint URLs
  autostart: false

# ── LLM Model Selection ──
llm:
  default_model: "gemma4:31b"        # Model used when none specified
  reranker_model: "gpt-oss:120b"     # Used for search result reranking
  scraper_model: "gemma4:31b"         # Used for web page summarization
  summary_model: "qwen3.5:397b"      # Used for content summarization
  fallback_model: "gemma4:31b"        # Used when requested model unavailable
  emulate_openai: true                # Enable /v1/chat/completions endpoint
  emulate_anthropic: true             # Enable /v1/messages proxy endpoint
  # available_models: [...]

# ── Fallback Provider (when Ollama Cloud is unavailable) ──
fallback:
  enabled: false
  name: "openai"                      # Display name
  base_url: "https://api.openai.com/v1"
  api_key: ""
  default_model: "gpt-4o"
  timeout: 60.0                       # Request timeout in seconds
  primary_timeout: 30.0               # Max seconds to wait for Ollama first
                                       # chunk before trying fallback
  stream_chunk_timeout: 180.0         # Max seconds between stream chunks
  max_tokens: 128000
  stream_fallback: true
  model_map: {}                        # ollama_model -> fallback_model mapping

# ── Search/Scrape Provider API Keys ──
providers:
  tavily:     { enabled: true }
  exa:        { enabled: true }
  searxng:    { enabled: true }
  firecrawl:  { enabled: true, require_api_key: false }
  serper:     { enabled: true }
  jina:       { enabled: true }
  cohere:     { enabled: true }
  brave:      { enabled: true }

# ── Smart Cache (BETA) ──
cache:
  beta_mode: false                    # Master switch — must be true to enable
  exact_cache_ttl: 600                # Seconds for exact-match response cache
  session_prefix_ttl: 3600            # Seconds for session prefix cache
  max_entries: 500
  dedup_enabled: true                 # Merge identical concurrent upstream calls
  session_prefix_enabled: true
  exact_cache_enabled: true
  min_prompt_chars: 50                # Don't cache tiny prompts

# ── Ollama Cloud Usage Tracking ──
usage:
  session_cookie: ""                   # __Secure-session cookie from ollama.com
  check_interval: 0                   # Auto-check interval (0 = disabled)
  redirect_on_full: false             # Route to fallback when quota near limit

Environment Variables

Variable	Description
OLLAMA_API_KEY	Ollama Cloud API key (takes precedence over config file)
GUANACO_CONFIG_DIR	Path to config directory (default ~/.guanaco)

---

Fallback Provider Setup

When Ollama Cloud is slow, rate-limited, or a requested model isn't available, Guanaco can automatically forward requests to a fallback OpenAI-compatible provider.

fallback:
  enabled: true
  name: "openai"
  base_url: "https://api.openai.com/v1"
  api_key: "sk-..."
  default_model: "gpt-4o"
  primary_timeout: 30.0                # Wait up to 30s for Ollama first chunk
  stream_chunk_timeout: 180.0          # Tolerate long reasoning pauses
  timeout: 60.0
  stream_fallback: true
  model_map:
    # Map specific Ollama models to different fallback models
    "qwen3:480b": "gpt-4o"
    "deepseek-v3.1:671b": "gpt-4o"

Or configure via the dashboard at Dashboard → Config → Fallback.

---

API Reference

LLM Router

POST /v1/chat/completions — OpenAI-compatible chat completions

curl -X POST http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -d '{
    "model": "gemma4:31b",
    "messages": [{"role": "user", "content": "Hello!"}],
    "stream": false
  }'

POST /v1/messages — Anthropic-compatible messages proxy

curl -X POST http://localhost:8080/v1/messages \
  -H "Content-Type: application/json" \
  -H "x-api-key: YOUR_API_KEY" \
  -H "anthropic-version: 2023-06-01" \
  -d '{
    "model": "gemma4:31b",
    "messages": [{"role": "user", "content": "Hello!"}],
    "max_tokens": 1024
  }'

Search APIs

All search providers are emulated at http://localhost:8080/<provider>/<endpoint>:

Provider	Endpoints	Notes
Tavily	/tavily/search	Tavily Search API compatible
Exa	/exa/search, /exa/findSimilar	Exa Search API compatible
SearXNG	/searxng/search	SearXNG API compatible
Firecrawl	/firecrawl/scrape, /firecrawl/search, /firecrawl/crawl, /firecrawl/extract	Firecrawl SDK v2 compatible
Serper	/serper/search, /serper/scrape	Serper API compatible
Jina	/jina/search, /jina/rerank	Jina API compatible
Cohere	/cohere/rerank	Cohere Rerank API compatible
Brave	/brave/search	Brave Search API compatible

Firecrawl SDK v2 paths (/v2/scrape, /v2/search, /v2/crawl, /v2/extract) are also supported directly.

Status & Utility Endpoints

Endpoint	Description
GET /health	Health check
GET /v1/models	List available models
GET /v1/usage	Ollama Cloud usage/quota
GET /api/ollama/status	Ollama Cloud connectivity
GET /api/ollama/models	Full model list with metadata

---

Docker Deployment

FROM python:3.12-slim
WORKDIR /app
COPY . .
RUN pip install -e .
EXPOSE 8080
ENV GUANACO_CONFIG_DIR=/data
VOLUME /data
CMD ["guanaco", "start", "--host", "0.0.0.0"]

docker build -t guanaco .
docker run -d -p 8080:8080 \
  -e OLLAMA_API_KEY=your_key \
  -v ~/.guanaco:/data \
  guanaco

---

systemd Deployment

sudo cp contrib/guanaco.service /etc/systemd/system/
sudo systemctl daemon-reload
sudo systemctl enable --now guanaco

Check status:

systemctl status guanaco
journalctl -u guanaco -f

Edit /etc/systemd/system/guanaco.service to set User, Group, install directory, and venv path as appropriate for your environment.

---

Backward Compatibility with Ollama Cloud Tools (OCT)

Guanaco is the successor to Ollama Cloud Tools (oct). The oct CLI is preserved as a drop-in alias:

oct start        # → runs guanaco start
oct status       # → runs guanaco status
oct models       # → runs guanaco models
oct config --show # → runs guanaco config --show

Config at ~/.oct/config.yaml is automatically read if ~/.guanaco/config.yaml doesn't exist. Update your scripts at your convenience — both commands work indefinitely.

---

Contributing

Contributions are welcome! Please open an issue or submit a pull request on the GitHub repository.

---

License

MIT — Copyright 2026 Guanaco Contributors