Commit graph

15 commits

Author SHA1 Message Date
Muhamad Aji Wibisono
42cc1b8fbe feat: applied coderabbit suggestions 2025-06-02 21:03:50 +07:00
Muhamad Aji Wibisono
d11b636113 feat: removed limit on message fetching 2025-06-02 20:26:14 +07:00
Muhamad Aji Wibisono
a0f9efdb67 fix: async thread on discord but 2025-06-02 19:22:04 +07:00
Muhamad Aji Wibisono
4b3c662478 feat: added discord indexer 2025-06-02 18:30:38 +07:00
DESKTOP-RTLN3BA\$punk
a870e07cbf Merge commit 'fd6da4c4723ab28fc40605d109a9ba66cbd364e3' into dev 2025-05-28 23:52:33 -07:00
DESKTOP-RTLN3BA\$punk
5411bac8e0 feat: Added content based hashing to prevent duplicates and fix resync issues 2025-05-28 23:52:00 -07:00
google-labs-jules[bot]
299bb35d8c Hi there, it's Jules! I've just finished making some improvements.
Here's a rundown of what I did:

Fix: Robust Slack rate limiting, error handling & GitHub org repos

This update delivers comprehensive improvements to Slack connector stability and enhances the GitHub connector.

**Slack Connector (`slack_history.py`, `connectors_indexing_tasks.py`):**
- I've implemented proactive delays (1.2s for `conversations.history`, 3s for `conversations.list` pagination) and `Retry-After` header handling for 429 rate limit errors across `conversations.list`, `conversations.history`, and `users.info` API calls.
- I'll now gracefully handle `not_in_channel` errors when fetching conversation history by logging a warning and skipping the channel.
- I've refactored channel info fetching: `get_all_channels` now returns richer channel data (including `is_member`, `is_private`).
- I've removed direct calls to `conversations.info` from `connectors_indexing_tasks.py`, using the richer data from `get_all_channels` instead, to prevent associated rate limits.
- I corrected a `SyntaxError` (non-printable character) in `slack_history.py`.
- I've enhanced logging for rate limit actions, delays, and errors.
- I've updated unit tests in `test_slack_history.py` to cover all new logic.

**GitHub Connector (`github_connector.py`):**
- I've modified `get_user_repositories` to fetch all repositories accessible by you (owned, collaborated, organization) by changing the API call parameter from `type='owner'` to `type='all'`.
- I've included unit tests in `test_github_connector.py` for this change.
2025-05-27 13:39:42 +00:00
DESKTOP-RTLN3BA\$punk
22da221ad5 feat: Shifted to RecursiveChunker and CodeChunker
- Codebase Q/A should be lot better now.
2025-04-29 23:02:07 -07:00
Adamsmith6300
f2f426d5eb merge conflicts 2025-04-16 21:34:51 -07:00
Adamsmith6300
ae8c74a5aa select repos when adding gh connector 2025-04-16 19:59:38 -07:00
DESKTOP-RTLN3BA\$punk
e0eb9d4b8b feat: Added Linear Connector 2025-04-15 23:10:35 -07:00
Adamsmith6300
bb198e38c0 add github connector, add alembic for db migrations, fix bug updating connectors 2025-04-13 13:56:22 -07:00
DESKTOP-RTLN3BA\$punk
24fd873ca7 fix: Fixed Slack Reindexing 2025-03-26 17:44:38 -07:00
DESKTOP-RTLN3BA\$punk
23da404177 fix: Fixed Notion Reindexing & Updation 2025-03-26 17:19:10 -07:00
DESKTOP-RTLN3BA\$punk
da23012970 feat: SurfSense v0.0.6 init 2025-03-14 18:53:14 -07:00