vrr/goose
2
0
Fork 0
mirror of https://github.com/block/goose.git synced 2026-04-28 03:29:36 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 32

Docs: Include step-by-step model configuration instructions for first… (#5239)

Browse source
This commit is contained in:
Emma Youndtsmith 2025-10-17 15:38:50 -05:00 • committed by GitHub
parent 5729d56a67
commit 2df2f783c9
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
4 changed files with 190 additions and 94 deletions
Download patch file Download diff file Expand all files Collapse all files

106
documentation/docs/getting-started/providers.md
View file

@ -10,7 +10,7 @@ import { ModelSelectionTip } from '@site/src/components/ModelSelectionTip';
# Supported LLM Providers
Goose is compatible with a wide range of LLM providers, allowing you to choose and integrate your preferred model.
goose is compatible with a wide range of LLM providers, allowing you to choose and integrate your preferred model.
:::tip Model Selection
<ModelSelectionTip/>
@ -33,8 +33,8 @@ Goose is compatible with a wide range of LLM providers, allowing you to choose a
| [Groq](https://groq.com/) | High-performance inference hardware and tools for LLMs. | `GROQ_API_KEY` |
| [LiteLLM](https://docs.litellm.ai/docs/) | LiteLLM proxy supporting multiple models with automatic prompt caching and unified API access. | `LITELLM_HOST`, `LITELLM_BASE_PATH` (optional), `LITELLM_API_KEY` (optional), `LITELLM_CUSTOM_HEADERS` (optional), `LITELLM_TIMEOUT` (optional) |
| [Ollama](https://ollama.com/) | Local model runner supporting Qwen, Llama, DeepSeek, and other open-source models. **Because this provider runs locally, you must first [download and run a model](#local-llms).** | `OLLAMA_HOST` |
| [Ramalama](https://ramalama.ai/) | Local model using native [OCI](https://opencontainers.org/) container runtimes, [CNCF](https://www.cncf.io/) tools, and supporting models as OCI artifacts. Ramalama API an compatible alternative to Ollama and can be used with the Goose Ollama provider. Supports Qwen, Llama, DeepSeek, and other open-source models. **Because this provider runs locally, you must first [download and run a model](#local-llms).** | `OLLAMA_HOST` |
| [OpenAI](https://platform.openai.com/api-keys) | Provides gpt-4o, o1, and other advanced language models. Also supports OpenAI-compatible endpoints (e.g., self-hosted LLaMA, vLLM, KServe). **o1-mini and o1-preview are not supported because Goose uses tool calling.** | `OPENAI_API_KEY`, `OPENAI_HOST` (optional), `OPENAI_ORGANIZATION` (optional), `OPENAI_PROJECT` (optional), `OPENAI_CUSTOM_HEADERS` (optional) |
| [Ramalama](https://ramalama.ai/) | Local model using native [OCI](https://opencontainers.org/) container runtimes, [CNCF](https://www.cncf.io/) tools, and supporting models as OCI artifacts. Ramalama API an compatible alternative to Ollama and can be used with the goose Ollama provider. Supports Qwen, Llama, DeepSeek, and other open-source models. **Because this provider runs locally, you must first [download and run a model](#local-llms).** | `OLLAMA_HOST` |
| [OpenAI](https://platform.openai.com/api-keys) | Provides gpt-4o, o1, and other advanced language models. Also supports OpenAI-compatible endpoints (e.g., self-hosted LLaMA, vLLM, KServe). **o1-mini and o1-preview are not supported because goose uses tool calling.** | `OPENAI_API_KEY`, `OPENAI_HOST` (optional), `OPENAI_ORGANIZATION` (optional), `OPENAI_PROJECT` (optional), `OPENAI_CUSTOM_HEADERS` (optional) |
| [OpenRouter](https://openrouter.ai/) | API gateway for unified access to various models with features like rate-limiting management. | `OPENROUTER_API_KEY` |
| [Snowflake](https://docs.snowflake.com/user-guide/snowflake-cortex/aisql#choosing-a-model) | Access the latest models using Snowflake Cortex services, including Claude models. **Requires a Snowflake account and programmatic access token (PAT)**. | `SNOWFLAKE_HOST`, `SNOWFLAKE_TOKEN` |
| [Tetrate Agent Router Service](https://router.tetrate.ai) | Unified API gateway for AI models including Claude, Gemini, GPT, open-weight models, and others. Supports PKCE authentication flow for secure API key generation. | `TETRATE_API_KEY`, `TETRATE_HOST` (optional) |
@ -43,7 +43,7 @@ Goose is compatible with a wide range of LLM providers, allowing you to choose a
## CLI Providers
Goose also supports special "pass-through" providers that work with existing CLI tools, allowing you to use your subscriptions instead of paying per token:
goose also supports special "pass-through" providers that work with existing CLI tools, allowing you to use your subscriptions instead of paying per token:
| Provider | Description | Requirements |
|-----------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
@ -58,10 +58,40 @@ CLI providers are cost-effective alternatives that use your existing subscriptio
## Configure Provider
To configure your chosen provider or see available options, visit the `Models` tab in the Goose Desktop or run `goose configure` in the CLI.
To configure your chosen provider or see available options, visit the `Models` tab in goose Desktop or run `goose configure` in the CLI.
<Tabs groupId="interface">
<TabItem value="ui" label="Goose Desktop" default>
<TabItem value="ui" label="goose Desktop" default>
**First-time users:**
On the welcome screen the first time you open goose, you have three options:
- **Automatic setup with [Tetrate Agent Router](https://tetrate.io/products/tetrate-agent-router-service)**
- **Automatic Setup with [OpenRouter](https://openrouter.ai/)**
- **Other Providers**
<Tabs groupId="setup">
<TabItem value="tetrate" label="Tetrate Agent Router" default>
We recommend starting with Tetrate Agent Router. Tetrate provides access to multiple AI models with built-in rate limiting and automatic failover.
:::info Free Credits Offer
You'll receive $10 in free credits the first time you automatically authenticate with Tetrate through goose. This offer is available to both new and existing Tetrate users.
:::
1. Choose `Automatic setup with Tetrate Agent Router`.
2. goose will open a browser window for you to authenticate with Tetrate, or create a new account if you don't have one already.
3. When you return to the goose desktop app, you're ready to begin your first session.
</TabItem>
<TabItem value="openrouter" label="OpenRouter">
1. Choose `Automatic setup with OpenRouter`.
2. goose will open a browser window for you to authenticate with OpenRouter, or create a new account if you don't have one already.
3. When you return to the goose desktop app, you're ready to begin your first session.
</TabItem>
<TabItem value="others" label="Other Providers">
1. If you have a specific provider you want to use with goose, and an API key from that provider, choose `Other Providers`.
2. Find the provider of your choice and click its `Configure` button. If you don't see your provider in the list, click `Add Custom Provider` at the bottom of the window.
3. Depending on your provider, you'll need to input your API Key, API Host, or other optional [parameters](#available-providers). Click the `Submit` button to authenticate and begin your first session.
</TabItem>
</Tabs>
**To update your LLM provider and API key:**
1. Click the <PanelLeft className="inline" size={16} /> button in the top-left to open the sidebar
2. Click the `Settings` button on the sidebar
@ -89,8 +119,8 @@ To configure your chosen provider or see available options, visit the `Models` t
3. Click the `Models` tab
4. Click `Reset Provider and Model` to clear your current settings and return to the welcome screen
</TabItem>
<TabItem value="cli" label="Goose CLI">
1. Run the following command:
<TabItem value="cli" label="goose CLI">
1. In your terminal, run the following command:
```sh
goose configure
@ -106,7 +136,7 @@ To configure your chosen provider or see available options, visit the `Models` t
│ ○ Add Extension
│ ○ Toggle Extensions
│ ○ Remove Extension
│ ○ Goose Settings
│ ○ goose Settings
└
```
3. Choose a model provider and press Enter.
@ -157,13 +187,13 @@ To configure your chosen provider or see available options, visit the `Models` t
│
◓ Checking your configuration...
└ Configuration saved successfully
```
```
</TabItem>
</Tabs>
## Using Custom OpenAI Endpoints
Goose supports using custom OpenAI-compatible endpoints, which is particularly useful for:
goose supports using custom OpenAI-compatible endpoints, which is particularly useful for:
- Self-hosted LLMs (e.g., LLaMA, Mistral) using vLLM or KServe
- Private OpenAI-compatible API servers
- Enterprise deployments requiring data governance and security compliance
@ -220,7 +250,7 @@ Goose supports using custom OpenAI-compatible endpoints, which is particularly u
### Setup Instructions
<Tabs groupId="interface">
<TabItem value="ui" label="Goose Desktop" default>
<TabItem value="ui" label="goose Desktop" default>
1. Click the <PanelLeft className="inline" size={16} /> button in the top-left to open the sidebar
2. Click the `Settings` button on the sidebar
3. Click the `Models` tab
@ -233,7 +263,7 @@ Goose supports using custom OpenAI-compatible endpoints, which is particularly u
- Project (for resource management)
7. Click `Submit`
</TabItem>
<TabItem value="cli" label="Goose CLI">
<TabItem value="cli" label="goose CLI">
1. Run `goose configure`
2. Select `Configure Providers`
3. Choose `OpenAI` as the provider
@ -249,19 +279,19 @@ Goose supports using custom OpenAI-compatible endpoints, which is particularly u
For enterprise deployments, you can pre-configure these values using environment variables or configuration files to ensure consistent governance across your organization.
:::
## Using Goose for Free
## Using goose for Free
Goose is a free and open source AI agent that you can start using right away, but not all supported [LLM Providers][providers] provide a free tier.
goose is a free and open source AI agent that you can start using right away, but not all supported [LLM Providers][providers] provide a free tier.
Below, we outline a couple of free options and how to get started with them.
:::warning Limitations
These free options are a great way to get started with Goose and explore its capabilities. However, you may need to upgrade your LLM for better performance.
These free options are a great way to get started with goose and explore its capabilities. However, you may need to upgrade your LLM for better performance.
:::
### Groq
Groq provides free access to open source models with high-speed inference. To use Groq with Goose, you need an API key from [Groq Console](https://console.groq.com/keys).
Groq provides free access to open source models with high-speed inference. To use Groq with goose, you need an API key from [Groq Console](https://console.groq.com/keys).
Groq offers several open source models that support tool calling:
- **moonshotai/kimi-k2-instruct** - Mixture-of-Experts model with 1 trillion parameters, optimized for agentic intelligence and tool use
@ -269,10 +299,10 @@ Groq offers several open source models that support tool calling:
- **gemma2-9b-it** - Google's Gemma 2 model with instruction tuning
- **llama-3.3-70b-versatile** - Meta's Llama 3.3 model for versatile applications
To set up Groq with Goose, follow these steps:
To set up Groq with goose, follow these steps:
<Tabs groupId="interface">
<TabItem value="ui" label="Goose Desktop" default>
<TabItem value="ui" label="goose Desktop" default>
**To update your LLM provider and API key:**
1. Click the <PanelLeft className="inline" size={16} /> button in the top-left to open the sidebar.
@ -283,7 +313,7 @@ To set up Groq with Goose, follow these steps:
6. Click `Configure`, enter your API key, and click `Submit`.
</TabItem>
<TabItem value="cli" label="Goose CLI">
<TabItem value="cli" label="goose CLI">
1. Run:
```sh
goose configure
@ -296,12 +326,12 @@ To set up Groq with Goose, follow these steps:
</Tabs>
### Google Gemini
Google Gemini provides a free tier. To start using the Gemini API with Goose, you need an API Key from [Google AI studio](https://aistudio.google.com/app/apikey).
Google Gemini provides a free tier. To start using the Gemini API with goose, you need an API Key from [Google AI studio](https://aistudio.google.com/app/apikey).
To set up Google Gemini with Goose, follow these steps:
To set up Google Gemini with goose, follow these steps:
<Tabs groupId="interface">
<TabItem value="ui" label="Goose Desktop" default>
<TabItem value="ui" label="goose Desktop" default>
**To update your LLM provider and API key:**
1. Click the <PanelLeft className="inline" size={16} /> button in the top-left to open the sidebar.
@ -312,7 +342,7 @@ To set up Google Gemini with Goose, follow these steps:
6. Click `Configure`, enter your API key, and click `Submit`.
</TabItem>
<TabItem value="cli" label="Goose CLI">
<TabItem value="cli" label="goose CLI">
1. Run:
```sh
goose configure
@ -347,10 +377,10 @@ To set up Google Gemini with Goose, follow these steps:
### Local LLMs
Goose is a local AI agent, and by using a local LLM, you keep your data private, maintain full control over your environment, and can work entirely offline without relying on cloud access. However, please note that local LLMs require a bit more set up before you can use one of them with Goose.
goose is a local AI agent, and by using a local LLM, you keep your data private, maintain full control over your environment, and can work entirely offline without relying on cloud access. However, please note that local LLMs require a bit more set up before you can use one of them with goose.
:::warning Limited Support for models without tool calling
Goose extensively uses tool calling, so models without it can only do chat completion. If using models without tool calling, all Goose [extensions must be disabled](/docs/getting-started/using-extensions#enablingdisabling-extensions).
goose extensively uses tool calling, so models without it can only do chat completion. If using models without tool calling, all goose [extensions must be disabled](/docs/getting-started/using-extensions#enablingdisabling-extensions).
:::
Here are some local providers we support:
@ -362,7 +392,7 @@ Here are some local providers we support:
1. [Download Ramalama](https://github.com/containers/ramalama?tab=readme-ov-file#install).
2. In a terminal, run any Ollama [model supporting tool-calling](https://ollama.com/search?c=tools) or [GGUF format HuggingFace Model](https://huggingface.co/search/full-text?q=%22tools+support%22+%2B+%22gguf%22&type=model):
The `--runtime-args="--jinja"` flag is required for Ramalama to work with the Goose Ollama provider.
The `--runtime-args="--jinja"` flag is required for Ramalama to work with the goose Ollama provider.
Example:
@ -370,7 +400,7 @@ Here are some local providers we support:
ramalama serve --runtime-args="--jinja" ollama://qwen2.5
```
3. In a separate terminal window, configure with Goose:
3. In a separate terminal window, configure with goose:
```sh
goose configure
@ -388,7 +418,7 @@ Here are some local providers we support:
└
```
5. Choose `Ollama` as the model provider since Ramalama is API compatible and can use the Goose Ollama provider
5. Choose `Ollama` as the model provider since Ramalama is API compatible and can use the goose Ollama provider
```
┌ goose-configure
@ -451,12 +481,12 @@ Here are some local providers we support:
```
:::tip Context Length
If you notice that Goose is having trouble using extensions or is ignoring [.goosehints](/docs/guides/using-goosehints), it is likely that the model's default context length of 2048 tokens is too low. Use `ramalama serve` to set the `--ctx-size, -c` option to a [higher value](https://github.com/containers/ramalama/blob/main/docs/ramalama-serve.1.md#--ctx-size--c).
If you notice that goose is having trouble using extensions or is ignoring [.goosehints](/docs/guides/using-goosehints), it is likely that the model's default context length of 2048 tokens is too low. Use `ramalama serve` to set the `--ctx-size, -c` option to a [higher value](https://github.com/containers/ramalama/blob/main/docs/ramalama-serve.1.md#--ctx-size--c).
:::
</TabItem>
<TabItem value="deepseek" label="DeepSeek-R1">
The native `DeepSeek-r1` model doesn't support tool calling, however, we have a [custom model](https://ollama.com/michaelneale/deepseek-r1-goose) you can use with Goose.
The native `DeepSeek-r1` model doesn't support tool calling, however, we have a [custom model](https://ollama.com/michaelneale/deepseek-r1-goose) you can use with goose.
:::warning
Note that this is a 70B model size and requires a powerful device to run smoothly.
@ -470,7 +500,7 @@ Here are some local providers we support:
ollama run michaelneale/deepseek-r1-goose
```
3. In a separate terminal window, configure with Goose:
3. In a separate terminal window, configure with goose:
```sh
goose configure
@ -555,7 +585,7 @@ Here are some local providers we support:
ollama run qwen2.5
```
3. In a separate terminal window, configure with Goose:
3. In a separate terminal window, configure with goose:
```sh
goose configure
@ -638,7 +668,7 @@ Here are some local providers we support:
```
:::tip Context Length
If you notice that Goose is having trouble using extensions or is ignoring [.goosehints](/docs/guides/using-goosehints), it is likely that the model's default context length of 4096 tokens is too low. Set the `OLLAMA_CONTEXT_LENGTH` environment variable to a [higher value](https://github.com/ollama/ollama/blob/main/docs/faq.md#how-can-i-specify-the-context-window-size).
If you notice that goose is having trouble using extensions or is ignoring [.goosehints](/docs/guides/using-goosehints), it is likely that the model's default context length of 4096 tokens is too low. Set the `OLLAMA_CONTEXT_LENGTH` environment variable to a [higher value](https://github.com/ollama/ollama/blob/main/docs/faq.md#how-can-i-specify-the-context-window-size).
:::
</TabItem>
@ -655,7 +685,7 @@ Here are some local providers we support:
docker model pull hf.co/unsloth/gemma-3n-e4b-it-gguf:q6_k
```
4. Configure Goose to use Docker Model Runner, using the OpenAI API compatible endpoint:
4. Configure goose to use Docker Model Runner, using the OpenAI API compatible endpoint:
```sh
goose configure
@ -718,7 +748,7 @@ Here are some local providers we support:
Docker model runner uses `/engines/llama.cpp/v1/chat/completions` for the base path.
9. Finally configure the model available in Docker Model Runner to be used by Goose: `hf.co/unsloth/gemma-3n-e4b-it-gguf:q6_k`
9. Finally configure the model available in Docker Model Runner to be used by goose: `hf.co/unsloth/gemma-3n-e4b-it-gguf:q6_k`
```
│
@ -735,7 +765,7 @@ Here are some local providers we support:
## Azure OpenAI Credential Chain
Goose supports two authentication methods for Azure OpenAI:
goose supports two authentication methods for Azure OpenAI:
1. **API Key Authentication** - Uses the `AZURE_OPENAI_API_KEY` for direct authentication
2. **Azure Credential Chain** - Uses Azure CLI credentials automatically without requiring an API key
@ -757,7 +787,7 @@ Beyond single-model setups, goose supports [multi-model configurations](/docs/gu
---
If you have any questions or need help with a specific provider, feel free to reach out to us on [Discord](https://discord.gg/block-opensource) or on the [Goose repo](https://github.com/block/goose).
If you have any questions or need help with a specific provider, feel free to reach out to us on [Discord](https://discord.gg/block-opensource) or on the [goose repo](https://github.com/block/goose).
[providers]: /docs/getting-started/providers