mirror of
https://github.com/supermemoryai/supermemory.git
synced 2026-05-17 12:20:04 +00:00
571 lines
17 KiB
Text
571 lines
17 KiB
Text
---
|
|
title: "GitHub Connector"
|
|
description: "Connect GitHub repositories to sync documentation files into your Supermemory knowledge base"
|
|
icon: "github"
|
|
---
|
|
|
|
Connect GitHub repositories to sync documentation files into your Supermemory knowledge base with OAuth authentication, webhook support, and automatic incremental syncing.
|
|
|
|
<Warning>
|
|
The GitHub connector requires a **Scale Plan** or **Enterprise Plan**.
|
|
</Warning>
|
|
|
|
## Quick Setup
|
|
|
|
### 1. Create GitHub Connection
|
|
|
|
<Tabs>
|
|
<Tab title="TypeScript">
|
|
```typescript
|
|
import Supermemory from 'supermemory';
|
|
|
|
const client = new Supermemory({
|
|
apiKey: process.env.SUPERMEMORY_API_KEY!
|
|
});
|
|
|
|
const connection = await client.connections.create('github', {
|
|
redirectUrl: 'https://yourapp.com/auth/github/callback',
|
|
containerTags: ['user-123', 'github-sync'],
|
|
documentLimit: 5000,
|
|
metadata: {
|
|
source: 'github',
|
|
team: 'engineering'
|
|
}
|
|
});
|
|
|
|
// Redirect user to GitHub OAuth
|
|
window.location.href = connection.authLink;
|
|
console.log('Auth expires in:', connection.expiresIn);
|
|
```
|
|
</Tab>
|
|
<Tab title="Python">
|
|
```python
|
|
from supermemory import Supermemory
|
|
import os
|
|
|
|
client = Supermemory(api_key=os.environ.get("SUPERMEMORY_API_KEY"))
|
|
|
|
connection = client.connections.create(
|
|
'github',
|
|
redirect_url='https://yourapp.com/auth/github/callback',
|
|
container_tags=['user-123', 'github-sync'],
|
|
document_limit=10000,
|
|
metadata={
|
|
'source': 'github',
|
|
'team': 'engineering'
|
|
}
|
|
)
|
|
|
|
# Redirect user to GitHub OAuth
|
|
print(f'Redirect to: {connection.auth_link}')
|
|
print(f'Expires in: {connection.expires_in}')
|
|
```
|
|
</Tab>
|
|
<Tab title="cURL">
|
|
```bash
|
|
curl -X POST "https://api.supermemory.ai/v3/connections/github" \
|
|
-H "Authorization: Bearer $SUPERMEMORY_API_KEY" \
|
|
-H "Content-Type: application/json" \
|
|
-d '{
|
|
"redirectUrl": "https://yourapp.com/auth/github/callback",
|
|
"containerTags": ["user-123", "github-sync"],
|
|
"documentLimit": 5000,
|
|
"metadata": {
|
|
"source": "github",
|
|
"team": "engineering"
|
|
}
|
|
}'
|
|
```
|
|
</Tab>
|
|
</Tabs>
|
|
|
|
<Note>
|
|
**OAuth Scopes:** The GitHub connector requires these scopes:
|
|
- `repo` - Access to private and public repositories
|
|
- `user:email` - Access to user's email address
|
|
- `admin:repo_hook` - Manage webhooks for incremental sync
|
|
</Note>
|
|
|
|
### 2. Handle OAuth Callback
|
|
|
|
After the user grants permissions, GitHub redirects to your callback URL. The connection is automatically established, and the user can now select which repositories to sync.
|
|
|
|
### 3. List and Configure Repositories
|
|
|
|
Unlike other connectors, GitHub requires repository selection before syncing begins. This gives your users control over which repositories to index.
|
|
|
|
<Note>
|
|
**Generic Endpoints:** GitHub uses the generic resource management endpoints (Get Resources and Configure Connection) that work for any provider supporting resource management. See [Managing Connection Resources](/memory-api/connectors/managing-resources) for detailed API documentation.
|
|
</Note>
|
|
|
|
<Tabs>
|
|
<Tab title="TypeScript">
|
|
```typescript
|
|
// List available repositories for the user
|
|
const repositories = await client.connections.github.listRepositories(
|
|
connectionId,
|
|
{
|
|
page: 1,
|
|
perPage: 100
|
|
}
|
|
);
|
|
|
|
// Display repositories in your UI
|
|
repositories.forEach(repo => {
|
|
console.log(`${repo.full_name} - ${repo.description}`);
|
|
console.log(`Private: ${repo.private}`);
|
|
console.log(`Default branch: ${repo.default_branch}`);
|
|
console.log(`Last updated: ${repo.updated_at}`);
|
|
});
|
|
|
|
// After user selects repositories, configure them
|
|
await client.connections.github.configure(connectionId, {
|
|
repositories: [
|
|
{
|
|
id: repo.id,
|
|
name: repo.full_name,
|
|
defaultBranch: repo.default_branch
|
|
}
|
|
]
|
|
});
|
|
|
|
console.log('Repository sync initiated');
|
|
```
|
|
</Tab>
|
|
<Tab title="Python">
|
|
```python
|
|
# List available repositories for the user
|
|
repositories = client.connections.github.list_repositories(
|
|
connection_id,
|
|
page=1,
|
|
per_page=100
|
|
)
|
|
|
|
# Display repositories in your UI
|
|
for repo in repositories:
|
|
print(f'{repo.full_name} - {repo.description}')
|
|
print(f'Private: {repo.private}')
|
|
print(f'Default branch: {repo.default_branch}')
|
|
print(f'Last updated: {repo.updated_at}')
|
|
|
|
# After user selects repositories, configure them
|
|
client.connections.github.configure(
|
|
connection_id,
|
|
repositories=[
|
|
{
|
|
'id': repo.id,
|
|
'name': repo.full_name,
|
|
'defaultBranch': repo.default_branch
|
|
}
|
|
]
|
|
)
|
|
|
|
print('Repository sync initiated')
|
|
```
|
|
</Tab>
|
|
<Tab title="cURL">
|
|
```bash
|
|
# List available repositories
|
|
curl -X GET "https://api.supermemory.ai/v3/connections/{connectionId}/resources?page=1&per_page=100" \
|
|
-H "Authorization: Bearer $SUPERMEMORY_API_KEY"
|
|
|
|
# Configure selected repositories
|
|
curl -X POST "https://api.supermemory.ai/v3/connections/{connectionId}/configure" \
|
|
-H "Authorization: Bearer $SUPERMEMORY_API_KEY" \
|
|
-H "Content-Type: application/json" \
|
|
-d '{
|
|
"resources": [
|
|
{
|
|
"id": 123456789,
|
|
"name": "your-org/documentation",
|
|
"defaultBranch": "main"
|
|
},
|
|
{
|
|
"id": 987654321,
|
|
"name": "your-org/api-docs",
|
|
"defaultBranch": "main"
|
|
}
|
|
]
|
|
}'
|
|
```
|
|
</Tab>
|
|
</Tabs>
|
|
|
|
<Warning>
|
|
**API-First Design:**
|
|
|
|
Supermemory provides the API endpoints to list and configure repositories. As a Supermemory customer, you need to build the UI in your application where your end-users can:
|
|
1. View their available GitHub repositories
|
|
2. Select which repositories to sync
|
|
3. Confirm the selection
|
|
|
|
This gives you complete control over the user experience and allows you to integrate repository selection seamlessly into your application's workflow.
|
|
</Warning>
|
|
|
|
## Supported Document Types
|
|
|
|
The GitHub connector syncs documentation and text files with the following extensions:
|
|
|
|
- **Markdown files**: `.md`, `.mdx`, `.markdown`
|
|
- **Text files**: `.txt`
|
|
- **reStructuredText**: `.rst`
|
|
- **AsciiDoc**: `.adoc`
|
|
- **Org-mode**: `.org`
|
|
|
|
Files are indexed as `github_markdown` document type in Supermemory.
|
|
|
|
<Note>
|
|
Only text-based documentation files are synced. Binary files, images, and code files (`.js`, `.py`, `.go`, etc.) are excluded by default to focus on searchable documentation content.
|
|
</Note>
|
|
|
|
## Incremental Sync with Webhooks
|
|
|
|
The GitHub connector automatically sets up webhooks for real-time incremental syncing. When files are pushed or deleted in configured repositories, Supermemory is notified immediately.
|
|
|
|
<Note>
|
|
**Batch Processing:** Webhook events are processed in batches with a 10-minute delay to optimize performance and prevent excessive syncing during rapid commits. This means changes pushed to your repository will be reflected in Supermemory within approximately 10 minutes.
|
|
</Note>
|
|
|
|
### How It Works
|
|
|
|
1. **Webhook Setup**: When you configure repositories, a webhook is automatically installed in each repository
|
|
2. **Push Events**: When commits are pushed to the default branch, changed documentation files are synced
|
|
3. **Delete Events**: When documentation files are deleted, they're removed from your Supermemory knowledge base
|
|
4. **Incremental Updates**: Only changed files are processed, keeping sync fast and efficient
|
|
|
|
### Webhook Security
|
|
|
|
Webhooks are secured using HMAC-SHA256 signature validation with constant-time comparison. Supermemory automatically validates that webhook events come from GitHub before processing them. Each repository gets a unique webhook secret for maximum security.
|
|
|
|
<Tabs>
|
|
<Tab title="TypeScript">
|
|
```typescript
|
|
// Check webhook status
|
|
const connection = await client.connections.get(connectionId);
|
|
|
|
console.log('Webhooks configured:', connection.metadata.webhooks?.length);
|
|
console.log('Last sync:', new Date(connection.metadata.lastSyncedAt));
|
|
console.log('Repositories:', connection.metadata.repositories);
|
|
```
|
|
</Tab>
|
|
<Tab title="Python">
|
|
```python
|
|
# Check webhook status
|
|
connection = client.connections.get(connection_id)
|
|
|
|
print(f'Webhooks configured: {len(connection.metadata.get("webhooks", []))}')
|
|
print(f'Last sync: {connection.metadata.get("lastSyncedAt")}')
|
|
print(f'Repositories: {connection.metadata.get("repositories")}')
|
|
```
|
|
</Tab>
|
|
<Tab title="cURL">
|
|
```bash
|
|
# Get connection details including webhook status
|
|
curl -X POST "https://api.supermemory.ai/v3/connections/list" \
|
|
-H "Authorization: Bearer $SUPERMEMORY_API_KEY" \
|
|
-H "Content-Type: application/json" \
|
|
-d '{}'
|
|
```
|
|
</Tab>
|
|
</Tabs>
|
|
|
|
## Connection Management
|
|
|
|
### List All Connections
|
|
|
|
<Tabs>
|
|
<Tab title="TypeScript">
|
|
```typescript
|
|
// List all GitHub connections for specific container tags
|
|
const connections = await client.connections.list({
|
|
containerTags: ['user-123'],
|
|
provider: 'github'
|
|
});
|
|
|
|
connections.forEach(conn => {
|
|
console.log(`Provider: ${conn.provider}`);
|
|
console.log(`ID: ${conn.id}`);
|
|
console.log(`Email: ${conn.email}`);
|
|
console.log(`Created: ${conn.createdAt}`);
|
|
console.log(`Document limit: ${conn.documentLimit}`);
|
|
console.log(`Repositories: ${conn.metadata.repositories?.length || 0}`);
|
|
console.log('---');
|
|
});
|
|
```
|
|
</Tab>
|
|
<Tab title="Python">
|
|
```python
|
|
# List all GitHub connections for specific container tags
|
|
connections = client.connections.list(
|
|
container_tags=['user-123'],
|
|
provider='github'
|
|
)
|
|
|
|
for conn in connections:
|
|
print(f'Provider: {conn.provider}')
|
|
print(f'ID: {conn.id}')
|
|
print(f'Email: {conn.email}')
|
|
print(f'Created: {conn.created_at}')
|
|
print(f'Document limit: {conn.document_limit}')
|
|
print(f'Repositories: {len(conn.metadata.get("repositories", []))}')
|
|
print('---')
|
|
```
|
|
</Tab>
|
|
<Tab title="cURL">
|
|
```bash
|
|
# List all GitHub connections for specific container tags
|
|
curl -X POST "https://api.supermemory.ai/v3/connections/list" \
|
|
-H "Authorization: Bearer $SUPERMEMORY_API_KEY" \
|
|
-H "Content-Type: application/json" \
|
|
-d '{
|
|
"containerTags": ["user-123"],
|
|
"provider": "github"
|
|
}'
|
|
```
|
|
</Tab>
|
|
</Tabs>
|
|
|
|
### Update Repository Configuration
|
|
|
|
You can update which repositories are synced at any time:
|
|
|
|
<Tabs>
|
|
<Tab title="TypeScript">
|
|
```typescript
|
|
// Add or remove repositories
|
|
await client.connections.github.configure(connectionId, {
|
|
repositories: [
|
|
{
|
|
id: 123456789,
|
|
name: 'your-org/documentation',
|
|
defaultBranch: 'main'
|
|
},
|
|
{
|
|
id: 987654321,
|
|
name: 'your-org/new-repo',
|
|
defaultBranch: 'develop' // Can specify different branch
|
|
}
|
|
]
|
|
});
|
|
|
|
console.log('Repository configuration updated');
|
|
```
|
|
</Tab>
|
|
<Tab title="Python">
|
|
```python
|
|
# Add or remove repositories
|
|
client.connections.github.configure(
|
|
connection_id,
|
|
repositories=[
|
|
{
|
|
'id': 123456789,
|
|
'name': 'your-org/documentation',
|
|
'defaultBranch': 'main'
|
|
},
|
|
{
|
|
'id': 987654321,
|
|
'name': 'your-org/new-repo',
|
|
'defaultBranch': 'develop' # Can specify different branch
|
|
}
|
|
]
|
|
)
|
|
|
|
print('Repository configuration updated')
|
|
```
|
|
</Tab>
|
|
<Tab title="cURL">
|
|
```bash
|
|
# Update repository configuration
|
|
curl -X POST "https://api.supermemory.ai/v3/connections/{connectionId}/configure" \
|
|
-H "Authorization: Bearer $SUPERMEMORY_API_KEY" \
|
|
-H "Content-Type: application/json" \
|
|
-d '{
|
|
"resources": [
|
|
{
|
|
"id": 123456789,
|
|
"name": "your-org/documentation",
|
|
"defaultBranch": "main"
|
|
},
|
|
{
|
|
"id": 987654321,
|
|
"name": "your-org/new-repo",
|
|
"defaultBranch": "develop"
|
|
}
|
|
]
|
|
}'
|
|
```
|
|
</Tab>
|
|
</Tabs>
|
|
|
|
<Note>
|
|
When you update the repository configuration:
|
|
- New repositories are added and synced immediately
|
|
- Removed repositories have their webhooks deleted
|
|
- Existing documents from removed repositories remain in Supermemory unless you delete them manually
|
|
</Note>
|
|
|
|
### Delete Connection
|
|
|
|
<Tabs>
|
|
<Tab title="TypeScript">
|
|
```typescript
|
|
// Delete by connection ID
|
|
const result = await client.connections.deleteByID(connectionId);
|
|
|
|
// Or delete by provider (requires container tags)
|
|
const result = await client.connections.deleteByProvider('github', {
|
|
containerTags: ['user-123']
|
|
});
|
|
|
|
console.log('Deleted connection:', result.id);
|
|
```
|
|
</Tab>
|
|
<Tab title="Python">
|
|
```python
|
|
# Delete by connection ID
|
|
result = client.connections.delete_by_id(connection_id)
|
|
|
|
# Or delete by provider (requires container tags)
|
|
result = client.connections.delete_by_provider(
|
|
provider='github',
|
|
container_tags=['user-123']
|
|
)
|
|
|
|
print(f'Deleted connection: {result.id}')
|
|
```
|
|
</Tab>
|
|
<Tab title="cURL">
|
|
```bash
|
|
# Delete by connection ID
|
|
curl -X DELETE "https://api.supermemory.ai/v3/connections/{connectionId}" \
|
|
-H "Authorization: Bearer $SUPERMEMORY_API_KEY"
|
|
```
|
|
</Tab>
|
|
</Tabs>
|
|
|
|
<Warning>
|
|
Deleting a GitHub connection will:
|
|
- Stop all future syncs from configured repositories
|
|
- Remove all webhooks from the repositories
|
|
- Revoke the OAuth authorization
|
|
- **Permanently delete all synced documents** from your Supermemory knowledge base (unless you pass `deleteDocuments=false` as a query parameter to keep them)
|
|
</Warning>
|
|
|
|
### Manual Sync
|
|
|
|
Trigger a manual synchronization for all configured repositories:
|
|
|
|
<Tabs>
|
|
<Tab title="TypeScript">
|
|
```typescript
|
|
// Trigger sync for GitHub connections
|
|
await client.connections.import('github');
|
|
|
|
// Trigger sync for specific container tags
|
|
await client.connections.import('github', {
|
|
containerTags: ['user-123']
|
|
});
|
|
|
|
console.log('Manual sync initiated');
|
|
```
|
|
</Tab>
|
|
<Tab title="Python">
|
|
```python
|
|
# Trigger sync for GitHub connections
|
|
client.connections.import_('github')
|
|
|
|
# Trigger sync for specific container tags
|
|
client.connections.import_(
|
|
'github',
|
|
container_tags=['user-123']
|
|
)
|
|
|
|
print('Manual sync initiated')
|
|
```
|
|
</Tab>
|
|
<Tab title="cURL">
|
|
```bash
|
|
# Trigger sync for all GitHub connections
|
|
curl -X POST "https://api.supermemory.ai/v3/connections/github/import" \
|
|
-H "Authorization: Bearer $SUPERMEMORY_API_KEY"
|
|
|
|
# Trigger sync for specific container tags
|
|
curl -X POST "https://api.supermemory.ai/v3/connections/github/import" \
|
|
-H "Authorization: Bearer $SUPERMEMORY_API_KEY" \
|
|
-H "Content-Type: application/json" \
|
|
-d '{
|
|
"containerTags": ["user-123"]
|
|
}'
|
|
|
|
# Response: {"message": "Manual sync initiated", "provider": "github"}
|
|
```
|
|
</Tab>
|
|
</Tabs>
|
|
|
|
## Advanced Configuration
|
|
|
|
### Custom OAuth Application
|
|
|
|
For white-label deployments or custom branding, configure your own GitHub OAuth app using the settings API:
|
|
|
|
<Tabs>
|
|
<Tab title="TypeScript">
|
|
```typescript
|
|
// Update organization settings with your GitHub OAuth app
|
|
await client.settings.update({
|
|
githubCustomKeyEnabled: true,
|
|
githubClientId: 'Iv1.1234567890abcdef',
|
|
githubClientSecret: 'your-github-client-secret'
|
|
});
|
|
|
|
// Get current settings
|
|
const settings = await client.settings.get();
|
|
console.log('GitHub custom key enabled:', settings.githubCustomKeyEnabled);
|
|
console.log('Client ID configured:', !!settings.githubClientId);
|
|
```
|
|
</Tab>
|
|
<Tab title="Python">
|
|
```python
|
|
# Update organization settings with your GitHub OAuth app
|
|
client.settings.update(
|
|
github_custom_key_enabled=True,
|
|
github_client_id='Iv1.1234567890abcdef',
|
|
github_client_secret='your-github-client-secret'
|
|
)
|
|
|
|
# Get current settings
|
|
settings = client.settings.get()
|
|
print(f'GitHub custom key enabled: {settings.github_custom_key_enabled}')
|
|
print(f'Client ID configured: {bool(settings.github_client_id)}')
|
|
```
|
|
</Tab>
|
|
<Tab title="cURL">
|
|
```bash
|
|
# Update organization settings
|
|
curl -X PATCH "https://api.supermemory.ai/v3/settings" \
|
|
-H "Authorization: Bearer $SUPERMEMORY_API_KEY" \
|
|
-H "Content-Type: application/json" \
|
|
-d '{
|
|
"githubCustomKeyEnabled": true,
|
|
"githubClientId": "Iv1.1234567890abcdef",
|
|
"githubClientSecret": "your-github-client-secret"
|
|
}'
|
|
|
|
# Get current settings
|
|
curl -X GET "https://api.supermemory.ai/v3/settings" \
|
|
-H "Authorization: Bearer $SUPERMEMORY_API_KEY"
|
|
```
|
|
</Tab>
|
|
</Tabs>
|
|
|
|
<Note>
|
|
**Setting up a GitHub OAuth App:**
|
|
|
|
1. Go to GitHub Settings → Developer settings → OAuth Apps
|
|
2. Click "New OAuth App"
|
|
3. Set Authorization callback URL to: `https://api.supermemory.ai/v3/connections/auth/callback/github`
|
|
4. Copy the Client ID and generate a Client Secret
|
|
5. Configure them in Supermemory using the settings API above
|
|
|
|
After configuration, all new GitHub connections will use your custom OAuth app instead of Supermemory's default app.
|
|
</Note>
|