…x cancel_all() TOCTOU
- Remove tree_queue property setter (backward-compat hack; all callers
already migrated to replace_tree_queue()); keep property getter only
- Update 2 remaining tests that still used direct assignment to use
replace_tree_queue()
- Remove _set_connected() 1-line wrapper on DiscordPlatform; assign
_connected directly
- Fix cancel_all() TOCTOU: hold self._lock for the full loop so newly
created trees cannot slip through between the snapshot and cancellation
---------
Co-authored-by: Claude <noreply@anthropic.com>
## Summary
Added NVIDIA NIM as a second transcription option ( alongside local
Whisper). This lets you transcribe voice notes using NVIDIA's cloud API
instead of running Whisper locally.
## What changed
- **Transcription**: Now supports the two backends
- Local Whisper: Free, runs on your GPU/CPU (existing)
- NVIDIA NIM: Cloud API via Riva gRPC (new)
- **Supported models**: 8 NVIDIA NIM models added (Parakeet variants for
different languages, Whisper Large V3)
---------
Co-authored-by: Alishahryar1 <alishahryar2@gmail.com>
- `max_concurrency` is now always an `int` (default 5) — `None`/unlimited
is no longer a valid state; omitting the env var uses the default
- `GlobalRateLimiter`: semaphore is always created; `concurrency_slot()`
no longer has None guards; log message always includes concurrency
- `ProviderConfig.max_concurrency`: `int = 5` (was `int | None = None`)
- `Settings.provider_max_concurrency`: `int = Field(default=5, ...)` —
setting env var to an invalid value (e.g. empty string) raises
- `.env.example`: uncommented `PROVIDER_MAX_CONCURRENCY=5`
- README: updated config table default from `—` to `5`
- Tests: removed `test_concurrency_slot_noop_when_not_configured`;
updated mock settings to use `5` instead of `None`
https://claude.ai/code/session_014mrF1WMNgmNjtPBuoQHsbg
The max_sessions cap in CLISessionManager was the only thing enforcing
a limit on concurrent CLI processes. Now that provider concurrency is
controlled at the streaming layer (PROVIDER_MAX_CONCURRENCY semaphore),
the CLI session pool cap is redundant and removed entirely.
Changes:
- cli/manager.py: remove max_sessions param, cap check, _cleanup_idle_sessions_unlocked, max_sessions from get_stats()
- config/settings.py: remove max_cli_sessions field
- api/app.py: remove max_sessions=settings.max_cli_sessions from CLISessionManager constructor
- messaging/handler.py: remove "Waiting for slot" status check; stats display no longer shows Max CLI
- .env.example: remove MAX_CLI_SESSIONS line
- tests/cli/test_cli.py: remove max_sessions args and assertion from manager tests
- tests/cli/test_cli_manager_edge_cases.py: remove two tests for cap/cleanup behavior
- tests/api/test_app_lifespan_and_errors.py: remove max_cli_sessions from all SimpleNamespace settings
- tests/config/test_config.py: remove max_cli_sessions isinstance assertion
- tests/conftest.py: remove max_sessions from mock stats
- tests/messaging/test_handler.py: merge slot/capacity tests into single new-conversation test; remove Max CLI assertion from stats test
- tests/messaging/test_handler_markdown_and_status_edges.py: remove "Waiting for slot" assertion; drop max_sessions from all stats mocks
https://claude.ai/code/session_014mrF1WMNgmNjtPBuoQHsbg
Adds max_concurrency cap to GlobalRateLimiter using asyncio.Semaphore.
A request now waits for a concurrency slot before the sliding window rate
limit check, so at most N streams are open to the provider simultaneously,
even when the rate window would allow more.
Changes:
- providers/rate_limit.py: max_concurrency param, _concurrency_sem, concurrency_slot() asynccontextmanager
- providers/openai_compat.py: pass max_concurrency to limiter; wrap execute_with_retry + stream iteration in concurrency_slot()
- providers/base.py: max_concurrency field on ProviderConfig
- config/settings.py: provider_max_concurrency setting (PROVIDER_MAX_CONCURRENCY env var, default None = unlimited)
- api/dependencies.py: pass provider_max_concurrency into all three provider ProviderConfig instantiations
- .env.example: document PROVIDER_MAX_CONCURRENCY (commented out)
- tests/providers/test_provider_rate_limit.py: 5 new tests covering concurrency limit enforcement, slot release on exception, noop when unconfigured
- tests/api/test_dependencies.py: add provider_max_concurrency=None to mock settings helper
https://claude.ai/code/session_014mrF1WMNgmNjtPBuoQHsbg
- Introduced message_thread_id to the IncomingMessage model for handling forum topic IDs in Telegram.
- Updated messaging platforms (Discord and Telegram) to accept and process message_thread_id in send_message methods.
- Modified message handlers to utilize message_thread_id when sending messages.
- Enhanced test cases to validate the integration of message_thread_id in message handling.
This change improves support for forum supergroups in Telegram and enhances message management across platforms.
- Implemented functionality to cancel pending voice transcriptions when a user replies with the /clear command.
- Updated the Telegram and Discord platform classes to manage pending voice messages, including registration and cancellation logic.
- Enhanced the message handler to delete associated messages and notify users when a voice note is cancelled.
- Added tests to ensure the cancellation feature works as expected during transcription.
- Updated transcription logic to utilize Hugging Face's Whisper models instead of faster-whisper.
- Introduced new model mapping and pipeline loading functions.
- Adjusted tests to reflect changes in the transcription process.
- Updated documentation in README, .env.example, and settings to align with the new implementation.
- Ensured compatibility with CUDA 13 and removed unnecessary dependencies.
- Validate whisper_device in Settings and _get_local_model
- Reject 'auto' with clear ValueError/ValidationError
- Update docs in config, .env.example, README
- Add tests for invalid device and valid cpu/cuda
Co-authored-by: Ali Khokhar <alishahryar2@gmail.com>
When NVIDIA_NIM_API_KEY or OPENROUTER_API_KEY is empty or not set,
the proxy forwarded requests without a valid Authorization header,
causing providers to return 403 with 'Header of type authorization
was missing'.
Now fail fast with HTTP 503 and a clear message telling users to add
the key to .env, with links to obtain keys.
Fixes#29
Co-authored-by: Ali Khokhar <alishahryar2@gmail.com>
- Introduced a new optional field `status_message_id` in the IncomingMessage model to track the status of voice note processing.
- Updated the Telegram and Discord platforms to utilize the `status_message_id` for editing status messages instead of sending new ones.
- Modified tests to assert the correct status message ID is used during voice note handling.
- Changed status message text from "Processing voice note..." to "Transcribing voice note..." for clarity.
- Implemented a queue message to indicate "Processing voice note..." for both Telegram and Discord platforms.
- Updated the Telegram platform to send a status message when handling voice notes.
- Enhanced the test for Telegram voice handling to verify the queue message is sent correctly.
- Introduced voice note handling for Discord and Telegram platforms.
- Added configuration options for voice note functionality in settings.py and .env.example.
- Updated README to include voice note instructions and configuration details.
- Implemented audio attachment processing and transcription using faster-whisper.
- Enabled voice note support through message handlers in both platforms.
- Add MessageTree.set_current_task() method
- Update tree_processor to use set_current_task instead of _current_task
- Move nim_settings out of ProviderConfig, pass only to NvidiaNimProvider
- Update api/dependencies and all tests
Co-authored-by: Ali Khokhar <alishahryar2@gmail.com>
- Create providers/openai_compat.py with shared streaming logic
- Refactor NvidiaNimProvider, OpenRouterProvider, LMStudioProvider to extend it
- OpenRouter overrides _handle_extra_reasoning for reasoning_details
- Update test patches to providers.openai_compat
Co-authored-by: Ali Khokhar <alishahryar2@gmail.com>
- Create providers/common/ with sse_builder, message_converter, think_parser,
heuristic_tool_parser, error_mapping
- Update nvidia_nim/utils and errors to re-export from common for backward compat
- Update all provider clients and tests to import from providers.common
- Remove duplicated files from nvidia_nim/utils/
Co-authored-by: Ali Khokhar <alishahryar2@gmail.com>
- Implemented handling of the `/clear` command to clear specific branches or entire trees based on message replies.
- Added tests for various scenarios of the clear command, including clearing branches, handling unknown replies, and clearing entire trees.
- Enhanced `TreeQueueManager` with methods to cancel branches and remove subtrees, ensuring proper state management in the session store.
- Updated `SessionStore` and `TreeRepository` to support removal of node mappings and trees, improving data integrity during clear operations.
- Introduced `plans_directory` parameter in `CLISessionManager`, `CLISession`, and updated related methods to handle plan files.
- Updated `api/app.py` to pass the plans directory to the CLI session manager.
- Added a new test in `test_cli.py` to verify that the `--settings plansDirectory` argument is included when starting a task with a specified plans directory.
Updated the README to include new timeout settings. Implemented these timeouts in the provider classes and added corresponding tests to ensure they are correctly passed to the client. Also included environment variable support for the new settings.
- Introduced `LMStudioProvider` to the provider system.
- Added a new fixture `lmstudio_provider` in `conftest.py` for testing.
- Updated `get_provider` function to handle `lmstudio` as a valid provider type.
- Enhanced README and `.env.example` to include LM Studio configuration details.
- Updated settings to accommodate LM Studio's base URL and provider type.
- Added tests to verify the functionality of the LM Studio provider.
- Replaced NVIDIA NIM and OpenRouter specific rate limit settings with a generic provider rate limit in settings, tests, and environment files.
- Updated README.md to reflect the new provider rate limit configuration.
- Adjusted tests to validate the new provider rate limit attributes.
