vrr/SurfSense
2
0
Fork 0
mirror of https://github.com/MODSetter/SurfSense.git synced 2025-09-02 02:29:08 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

Merge pull request #165 from MODSetter/dev

Browse source 
chore updated docs
This commit is contained in:
Rohan Verma 2025-06-09 16:37:31 -07:00 • committed by GitHub
commit 38a2f16413
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
4 changed files with 26 additions and 81 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

8
README.md
View file

@ -147,6 +147,10 @@ Before installation, make sure to complete the [prerequisite setup steps](https:
## Screenshots ## Screenshots
**Research Agent**
![updated_researcher](https://github.com/user-attachments/assets/e22c5d86-f511-4c72-8c50-feba0c1561b4)
**Search Spaces** **Search Spaces**
![search_spaces](https://github.com/user-attachments/assets/e254c38c-f937-44b6-9e9d-770db583d099) ![search_spaces](https://github.com/user-attachments/assets/e254c38c-f937-44b6-9e9d-770db583d099)
@ -154,10 +158,6 @@ Before installation, make sure to complete the [prerequisite setup steps](https:
**Manage Documents** **Manage Documents**
![documents](https://github.com/user-attachments/assets/7001e306-eb06-4009-89c6-8fadfdc3fc4d) ![documents](https://github.com/user-attachments/assets/7001e306-eb06-4009-89c6-8fadfdc3fc4d)
**Research Agent**
![researcher](https://github.com/user-attachments/assets/fda3e61f-f936-4b66-b565-d84edde44a67)
**Podcast Agent** **Podcast Agent**
![podcasts](https://github.com/user-attachments/assets/6cb82ffd-9e14-4172-bc79-67faf34c4c1c) ![podcasts](https://github.com/user-attachments/assets/6cb82ffd-9e14-4172-bc79-67faf34c4c1c)

8
surfsense_backend/.env.example
View file

@ -45,11 +45,3 @@ LANGSMITH_API_KEY=lsv2_pt_.....
LANGSMITH_PROJECT=surfsense LANGSMITH_PROJECT=surfsense
# FAST_LLM=openai/gpt-4o-mini
# STRATEGIC_LLM=openai/gpt-4o
# LONG_CONTEXT_LLM=gemini/gemini-2.0-flash
# FAST_LLM=ollama/gemma3:12b
# STRATEGIC_LLM=ollama/deepseek-r1:8b
# LONG_CONTEXT_LLM=ollama/deepseek-r1:8b

45
surfsense_web/content/docs/docker-installation.mdx
View file

@ -83,37 +83,25 @@ Before you begin, ensure you have:
| -------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | -------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| DATABASE_URL | PostgreSQL connection string (e.g., `postgresql+asyncpg://postgres:postgres@localhost:5432/surfsense`) | | DATABASE_URL | PostgreSQL connection string (e.g., `postgresql+asyncpg://postgres:postgres@localhost:5432/surfsense`) |
| SECRET_KEY | JWT Secret key for authentication (should be a secure random string) | | SECRET_KEY | JWT Secret key for authentication (should be a secure random string) |
| AUTH_TYPE | Authentication method: `GOOGLE` for OAuth with Google, `LOCAL` for email/password authentication |
| NEXT_FRONTEND_URL | URL where your frontend application is hosted (e.g., `http://localhost:3000`) | | NEXT_FRONTEND_URL | URL where your frontend application is hosted (e.g., `http://localhost:3000`) |
| EMBEDDING_MODEL | Name of the embedding model (e.g., `openai://text-embedding-ada-002`, `anthropic://claude-v1`, `mixedbread-ai/mxbai-embed-large-v1`) | | AUTH_TYPE | Authentication method: `GOOGLE` for OAuth with Google, `LOCAL` for email/password authentication |
| GOOGLE_OAUTH_CLIENT_ID | (Optional) Client ID from Google Cloud Console (required if AUTH_TYPE=GOOGLE) |
| GOOGLE_OAUTH_CLIENT_SECRET | (Optional) Client secret from Google Cloud Console (required if AUTH_TYPE=GOOGLE) |
| EMBEDDING_MODEL | Name of the embedding model (e.g., `mixedbread-ai/mxbai-embed-large-v1`) |
| RERANKERS_MODEL_NAME | Name of the reranker model (e.g., `ms-marco-MiniLM-L-12-v2`) | | RERANKERS_MODEL_NAME | Name of the reranker model (e.g., `ms-marco-MiniLM-L-12-v2`) |
| RERANKERS_MODEL_TYPE | Type of reranker model (e.g., `flashrank`) | | RERANKERS_MODEL_TYPE | Type of reranker model (e.g., `flashrank`) |
| FAST_LLM | LiteLLM routed smaller, faster LLM (e.g., `openai/gpt-4o-mini`, `ollama/deepseek-r1:8b`) | | TTS_SERVICE | Text-to-Speech API provider for Podcasts (e.g., `openai/tts-1`). See [supported providers](https://docs.litellm.ai/docs/text_to_speech#supported-providers) |
| STRATEGIC_LLM | LiteLLM routed advanced LLM for complex tasks (e.g., `openai/gpt-4o`, `ollama/gemma3:12b`) | | TTS_SERVICE_API_KEY | API key for the Text-to-Speech service |
| LONG_CONTEXT_LLM | LiteLLM routed LLM for longer context windows (e.g., `gemini/gemini-2.0-flash`, `ollama/deepseek-r1:8b`) | | TTS_SERVICE_API_BASE | (Optional) Custom API base URL for the Text-to-Speech service |
| STT_SERVICE | Speech-to-Text API provider for Podcasts (e.g., `openai/whisper-1`). See [supported providers](https://docs.litellm.ai/docs/audio_transcription#supported-providers) |
| STT_SERVICE_API_KEY | API key for the Speech-to-Text service |
| STT_SERVICE_API_BASE | (Optional) Custom API base URL for the Speech-to-Text service |
| FIRECRAWL_API_KEY | API key for Firecrawl service for web crawling |
| ETL_SERVICE | Document parsing service: `UNSTRUCTURED` (supports 34+ formats) or `LLAMACLOUD` (supports 50+ formats including legacy document types) | | ETL_SERVICE | Document parsing service: `UNSTRUCTURED` (supports 34+ formats) or `LLAMACLOUD` (supports 50+ formats including legacy document types) |
| UNSTRUCTURED_API_KEY | API key for Unstructured.io service for document parsing (required if ETL_SERVICE=UNSTRUCTURED) | | UNSTRUCTURED_API_KEY | API key for Unstructured.io service for document parsing (required if ETL_SERVICE=UNSTRUCTURED) |
| LLAMA_CLOUD_API_KEY | API key for LlamaCloud service for document parsing (required if ETL_SERVICE=LLAMACLOUD) | | LLAMA_CLOUD_API_KEY | API key for LlamaCloud service for document parsing (required if ETL_SERVICE=LLAMACLOUD) |
| FIRECRAWL_API_KEY | API key for Firecrawl service for web crawling |
| TTS_SERVICE | Text-to-Speech API provider for Podcasts (e.g., `openai/tts-1`, `azure/neural`, `vertex_ai/`). See [supported providers](https://docs.litellm.ai/docs/text_to_speech#supported-providers) |
| STT_SERVICE | Speech-to-Text API provider for Podcasts (e.g., `openai/whisper-1`). See [supported providers](https://docs.litellm.ai/docs/audio_transcription#supported-providers) |
Include API keys for your chosen LLM providers:
| ENV VARIABLE | DESCRIPTION |
|--------------------|-----------------------------------------------------------------------------|
| `OPENAI_API_KEY` | Required if using OpenAI models |
| `GEMINI_API_KEY` | Required if using Google Gemini models |
| `ANTHROPIC_API_KEY`| Required if using Anthropic models |
### Google OAuth Configuration (if AUTH_TYPE=GOOGLE)
| ENV VARIABLE | DESCRIPTION |
|----------------------------|-----------------------------------------------------------------------------|
| `GOOGLE_OAUTH_CLIENT_ID` | Client ID from Google Cloud Console |
| `GOOGLE_OAUTH_CLIENT_SECRET` | Client secret from Google Cloud Console |
**Optional Backend LangSmith Observability:** **Optional Backend LangSmith Observability:**
| ENV VARIABLE | DESCRIPTION | | ENV VARIABLE | DESCRIPTION |
|--------------|-------------| |--------------|-------------|
@ -122,17 +110,6 @@ Include API keys for your chosen LLM providers:
| LANGSMITH_API_KEY | Your LangSmith API key | | LANGSMITH_API_KEY | Your LangSmith API key |
| LANGSMITH_PROJECT | LangSmith project name (e.g., `surfsense`) | | LANGSMITH_PROJECT | LangSmith project name (e.g., `surfsense`) |
**Optional Backend LiteLLM API Base URLs:**
| ENV VARIABLE | DESCRIPTION |
|--------------|-------------|
| FAST_LLM_API_BASE | Custom API base URL for the fast LLM |
| STRATEGIC_LLM_API_BASE | Custom API base URL for the strategic LLM |
| LONG_CONTEXT_LLM_API_BASE | Custom API base URL for the long context LLM |
| TTS_SERVICE_API_BASE | Custom API base URL for the Text-to-Speech (TTS) service |
| STT_SERVICE_API_BASE | Custom API base URL for the Speech-to-Text (STT) service |
For other LLM providers, refer to the [LiteLLM documentation](https://docs.litellm.ai/docs/providers).
### Frontend Environment Variables ### Frontend Environment Variables
| ENV VARIABLE | DESCRIPTION | | ENV VARIABLE | DESCRIPTION |

46
surfsense_web/content/docs/manual-installation.mdx
View file

@ -53,38 +53,23 @@ Edit the `.env` file and set the following variables:
| -------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | -------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| DATABASE_URL | PostgreSQL connection string (e.g., `postgresql+asyncpg://postgres:postgres@localhost:5432/surfsense`) | | DATABASE_URL | PostgreSQL connection string (e.g., `postgresql+asyncpg://postgres:postgres@localhost:5432/surfsense`) |
| SECRET_KEY | JWT Secret key for authentication (should be a secure random string) | | SECRET_KEY | JWT Secret key for authentication (should be a secure random string) |
| AUTH_TYPE | Authentication method: `GOOGLE` for OAuth with Google, `LOCAL` for email/password authentication |
| NEXT_FRONTEND_URL | URL where your frontend application is hosted (e.g., `http://localhost:3000`) | | NEXT_FRONTEND_URL | URL where your frontend application is hosted (e.g., `http://localhost:3000`) |
| EMBEDDING_MODEL | Name of the embedding model (e.g., `openai://text-embedding-ada-002`, `anthropic://claude-v1`, `mixedbread-ai/mxbai-embed-large-v1`) | | AUTH_TYPE | Authentication method: `GOOGLE` for OAuth with Google, `LOCAL` for email/password authentication |
| GOOGLE_OAUTH_CLIENT_ID | (Optional) Client ID from Google Cloud Console (required if AUTH_TYPE=GOOGLE) |
| GOOGLE_OAUTH_CLIENT_SECRET | (Optional) Client secret from Google Cloud Console (required if AUTH_TYPE=GOOGLE) |
| EMBEDDING_MODEL | Name of the embedding model (e.g., `mixedbread-ai/mxbai-embed-large-v1`) |
| RERANKERS_MODEL_NAME | Name of the reranker model (e.g., `ms-marco-MiniLM-L-12-v2`) | | RERANKERS_MODEL_NAME | Name of the reranker model (e.g., `ms-marco-MiniLM-L-12-v2`) |
| RERANKERS_MODEL_TYPE | Type of reranker model (e.g., `flashrank`) | | RERANKERS_MODEL_TYPE | Type of reranker model (e.g., `flashrank`) |
| FAST_LLM | LiteLLM routed smaller, faster LLM (e.g., `openai/gpt-4o-mini`, `ollama/deepseek-r1:8b`) | | TTS_SERVICE | Text-to-Speech API provider for Podcasts (e.g., `openai/tts-1`). See [supported providers](https://docs.litellm.ai/docs/text_to_speech#supported-providers) |
| STRATEGIC_LLM | LiteLLM routed advanced LLM for complex tasks (e.g., `openai/gpt-4o`, `ollama/gemma3:12b`) | | TTS_SERVICE_API_KEY | API key for the Text-to-Speech service |
| LONG_CONTEXT_LLM | LiteLLM routed LLM for longer context windows (e.g., `gemini/gemini-2.0-flash`, `ollama/deepseek-r1:8b`) | | TTS_SERVICE_API_BASE | (Optional) Custom API base URL for the Text-to-Speech service |
| STT_SERVICE | Speech-to-Text API provider for Podcasts (e.g., `openai/whisper-1`). See [supported providers](https://docs.litellm.ai/docs/audio_transcription#supported-providers) |
| STT_SERVICE_API_KEY | API key for the Speech-to-Text service |
| STT_SERVICE_API_BASE | (Optional) Custom API base URL for the Speech-to-Text service |
| FIRECRAWL_API_KEY | API key for Firecrawl service for web crawling |
| ETL_SERVICE | Document parsing service: `UNSTRUCTURED` (supports 34+ formats) or `LLAMACLOUD` (supports 50+ formats including legacy document types) | | ETL_SERVICE | Document parsing service: `UNSTRUCTURED` (supports 34+ formats) or `LLAMACLOUD` (supports 50+ formats including legacy document types) |
| UNSTRUCTURED_API_KEY | API key for Unstructured.io service for document parsing (required if ETL_SERVICE=UNSTRUCTURED) | | UNSTRUCTURED_API_KEY | API key for Unstructured.io service for document parsing (required if ETL_SERVICE=UNSTRUCTURED) |
| LLAMA_CLOUD_API_KEY | API key for LlamaCloud service for document parsing (required if ETL_SERVICE=LLAMACLOUD) | | LLAMA_CLOUD_API_KEY | API key for LlamaCloud service for document parsing (required if ETL_SERVICE=LLAMACLOUD) |
| FIRECRAWL_API_KEY | API key for Firecrawl service for web crawling |
| TTS_SERVICE | Text-to-Speech API provider for Podcasts (e.g., `openai/tts-1`, `azure/neural`, `vertex_ai/`). See [supported providers](https://docs.litellm.ai/docs/text_to_speech#supported-providers) |
| STT_SERVICE | Speech-to-Text API provider for Podcasts (e.g., `openai/whisper-1`). See [supported providers](https://docs.litellm.ai/docs/audio_transcription#supported-providers) |
Include API keys for your chosen LLM providers:
| ENV VARIABLE | DESCRIPTION |
|--------------------|-----------------------------------------------------------------------------|
| `OPENAI_API_KEY` | Required if using OpenAI models |
| `GEMINI_API_KEY` | Required if using Google Gemini models |
| `ANTHROPIC_API_KEY`| Required if using Anthropic models |
For other providers, refer to the [LiteLLM documentation](https://docs.litellm.ai/docs/providers)
### Google OAuth Configuration (if AUTH_TYPE=GOOGLE)
| ENV VARIABLE | DESCRIPTION |
|----------------------------|-----------------------------------------------------------------------------|
| `GOOGLE_OAUTH_CLIENT_ID` | Client ID from Google Cloud Console |
| `GOOGLE_OAUTH_CLIENT_SECRET` | Client secret from Google Cloud Console |
**Optional Backend LangSmith Observability:** **Optional Backend LangSmith Observability:**
@ -95,15 +80,6 @@ For other providers, refer to the [LiteLLM documentation](https://docs.litellm.a
| LANGSMITH_API_KEY | Your LangSmith API key | | LANGSMITH_API_KEY | Your LangSmith API key |
| LANGSMITH_PROJECT | LangSmith project name (e.g., `surfsense`) | | LANGSMITH_PROJECT | LangSmith project name (e.g., `surfsense`) |
**Optional Backend LiteLLM API Base URLs:**
| ENV VARIABLE | DESCRIPTION |
|--------------|-------------|
| FAST_LLM_API_BASE | Custom API base URL for the fast LLM |
| STRATEGIC_LLM_API_BASE | Custom API base URL for the strategic LLM |
| LONG_CONTEXT_LLM_API_BASE | Custom API base URL for the long context LLM |
| TTS_SERVICE_API_BASE | Custom API base URL for the Text-to-Speech (TTS) service |
| STT_SERVICE_API_BASE | Custom API base URL for the Speech-to-Text (STT) service |
### 2. Install Dependencies ### 2. Install Dependencies
Install the backend dependencies using `uv`: Install the backend dependencies using `uv`: