Commit graph

3966 commits

Author SHA1 Message Date
MODSetter
b7ad5ecfff feat(models): drop catalogue models that cannot serve a turn
Passing OpenRouter's metadata filters does not prove a model can complete a
turn through this harness, so dead-but-advertised models kept reaching users
and failing mid-stream. A new model_compatibility table holds one verdict per
model, a weekly Celery sweep re-probes stale entries, and catalogue
generation reads the blocked ids back so those models are never offered.
Also excludes :batch variants, which advertise full chat metadata and then
reject completions outright.

Auto-pin now falls through to Auto when an explicitly selected global config
is absent from the current worker's catalogue. Returning it handed
load_llm_bundle an id it could not resolve, ending the turn with SERVER_ERROR
on one of the four uvicorn workers while the identical request succeeded on
the others. Catalogue refresh also re-materializes GLOBAL_MODELS, which
previously kept its boot snapshot until restart.

On the web side a single dropped connection surfaced as one toast per
in-flight query and dozens of PostHog exceptions, with no retry button.
Network errors now share a toast id, are deduped per endpoint before capture,
carry host and online context, and count as retryable.

Consolidates the model filter helpers duplicated between the model list and
OpenRouter services into openrouter_model_normalizer, renames OoxmlDefect to
OoxmlError, and includes a ruff format and import-order pass.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-08-14 18:59:11 +05:30
Anish Sarkar
0aecab70c3 Merge remote-tracking branch 'upstream/dev' into fix/artifacts-sandbox-ui 2026-08-14 12:58:20 +05:30
Anish Sarkar
f3adc06954 refactor(artifacts): update artifact card layout and loading state styling 2026-08-14 12:32:20 +05:30
Anish Sarkar
35dcab3621 fix(artifacts): improve layout and overflow handling in artifact card and library components
- Updated `ArtifactCard` to ensure proper overflow handling and prevent layout issues with long text.
- Adjusted styles in `ArtifactsLibrary` for better responsiveness and to manage overflow in the grid layout.
2026-08-14 12:25:53 +05:30
Anish Sarkar
7734bc03ac feat(artifacts): integrate artifact panel opening in document preview 2026-08-14 12:19:16 +05:30
Anish Sarkar
ffd27e8373 feat(artifacts): enhance artifact panel state management and UI interactions
- Added `selectedCardToolCallId` to the artifact panel state for improved tracking of selected artifacts.
- Updated `openArtifactPanelAtom` to accept `selectedCardToolCallId` when opening the panel.
- Modified `ArtifactCard` component to pass `toolCallId` when opening the artifact panel, ensuring correct selection highlighting.
- Refactored `scroll-to-artifact` logic to utilize the new `selectedCardToolCallId` for better artifact navigation.
- Adjusted styles in the artifact card to visually indicate the selected state.
2026-08-14 12:06:34 +05:30
Anish Sarkar
576b4725d7 feat(chat-artifacts): enhance artifact handling with deep linking and loading states
- Introduced `useArtifactDeepLink` hook to manage deep linking for artifacts in chat.
- Updated `useSyncChatArtifacts` to return loading states and artifacts for better UI feedback.
- Added `ThreadMessagesSkeletonBody` component for improved loading experience in the thread view.
- Refactored `ArtifactCard` to support deep linking and streamlined artifact display logic.
- Removed unused media viewer dialog component to simplify the artifacts library UI.
2026-08-14 11:48:10 +05:30
Anish Sarkar
9197754c8f feat(documents): implement enhanced document type filtering with search and mobile support
- Introduced a new `DocumentTypeFilterList` component for improved filtering of document types.
- Added search functionality to filter document types based on user input.
- Implemented scroll position tracking to enhance user experience in the filter list.
- Updated `DocumentsFilters` to integrate the new filtering component and handle mobile responsiveness.
- Refactored existing state management for clarity and efficiency.
2026-08-14 01:46:12 +05:30
Anish Sarkar
0841c988f7 feat(documents): add artifact icon support in document type icon component
- Introduced support for displaying an `ArtifactFormatIcon` when the document type is "ARTIFACT".
- Updated the `FORMAT_META` to include a new presentation-only key for artifacts, enhancing the visual representation in the UI.
- Ensured consistent handling of document types across the application.
2026-08-14 01:39:10 +05:30
Anish Sarkar
54a0029a33 feat(artifacts): enhance artifact metadata handling and introduce format icons
- Added `document_id` to artifact listing for improved metadata association.
- Introduced `ArtifactFormatIcon` and `ArtifactFormatLabel` components for better visual representation of artifact formats.
- Updated various components to utilize new format handling, ensuring consistent display across the application.
- Refactored artifact-related schemas and hooks to accommodate new format data.
- Enhanced unit tests to cover new functionality and ensure robustness.
2026-08-14 01:34:50 +05:30
Anish Sarkar
1210e4fb41 refactor(artifacts): enhance artifact handling and update schemas
- Updated `ArtifactViewerContent` to conditionally display download filename based on workspace validity.
- Added `refetchOnWindowFocus` option to artifact query for improved data freshness.
- Replaced `documentStatus` with a Zod enum for `indexing_status` in artifact schemas.
- Refactored `collectArtifacts` to return `ArtifactCandidate` and adjusted related functions for better artifact management.
- Removed unused imports and streamlined the `ArtifactRow` component for clarity.
2026-08-14 00:50:10 +05:30
CREDO23
189d0e6381 Merge remote-tracking branch 'upstream/dev' into feat/media-artifact-alignment 2026-08-13 21:11:39 +02:00
CREDO23
0a2c856288 feat(podcasts): align delivery to the Artifact model
Rename podcasts -> podcast_runs and treat the run row as job state
only: delivered audio and its markdown live in the Artifact, so the
row drops storage_backend/storage_key/file_location and gains
artifact_id. mark_ready commits status and artifact_id atomically,
preserving the READY <-> artifact_id invariant that Zero publishes.

Public shares stream audio through the artifact content route and get
artifact_id + workspace_id injected at snapshot time; the authenticated
player and library read artifactId from Zero. Retire the legacy
authenticated stream route, the podcast storage helpers, and the REST
list endpoint.
2026-08-13 20:52:59 +02:00
Anish Sarkar
44da47a665 refactor(download-button): remove spinner and adjust layout in download button component 2026-08-13 23:57:18 +05:30
Anish Sarkar
53bcc19ed8 feat(artifacts): add thread scoping to artifact listing and enhance query handling
- Updated `list_artifacts` to accept an optional `thread_id` parameter for filtering artifacts by thread.
- Modified the query to conditionally include the `thread_id` in the selection criteria.
- Enhanced frontend components to support thread-specific artifact fetching and display.
- Added unit tests to verify the new functionality for scoping artifacts to a specific thread.
2026-08-13 23:49:38 +05:30
Anish Sarkar
536126a564 refactor(artifacts-panel): replace Boxes icon with Shapes and update heading style 2026-08-13 22:55:23 +05:30
CREDO23
f89c60bdc3 feat(video): align video presentations to the Artifact model
Video now delivers into an Artifact like image and podcast. The run row
becomes a lifecycle record and the Remotion payload lives in the Artifact.

- run carries artifact_id; the tool returns artifact_id + workspace_id and
  the frontend renders from the Artifact
- public shares serve video and slide audio through artifact-scoped routes
  behind the snapshot allowlist, replacing the broken audio_file path
- purge_document_blobs collects slide audio storage keys so deleting a video
  Artifact no longer orphans them
- backfill_video_artifacts.py converts legacy rows and repoints chat and
  snapshot payloads (run before the slim migration)
- migrations: 179 adds artifact_id (expand); 180 renames the table to
  video_presentation_runs, drops slides/scene_codes, adds error, and
  publishes to Zero (contract). 180 aborts if any delivered run still lacks
  an Artifact, so the backfill can't be skipped
- publish video_presentation_runs to Zero so in-flight and failed runs reach
  the library by push; retire the legacy REST routes, schema, and poller
2026-08-13 18:59:07 +02:00
Anish Sarkar
e8556c1ae3 refactor(artifact-download-button): remove spinner during download state 2026-08-13 22:17:49 +05:30
Anish Sarkar
1de0aa384d feat(xlsx-viewer): implement spreadsheet grid for improved Excel file preview
- Added a new `SpreadsheetGrid` component to render Excel sheets using `react-data-grid`.
- Enhanced the viewer to display worksheets with dynamic column and row generation.
- Updated loading and error handling messages for better user experience.
- Refactored existing table structure to utilize the new grid component for better performance and styling.
2026-08-13 22:17:16 +05:30
Anish Sarkar
52b9089034 feat(dependencies): add react-data-grid version 7.0.0-beta.61 to package.json and update pnpm-lock.yaml 2026-08-13 22:14:15 +05:30
Anish Sarkar
5ba32740c4 fix(Dockerfile): set CI environment variable during development build 2026-08-13 22:10:09 +05:30
Thierry CH.
13feaff677
Merge pull request #1678 from CREDO23/feat/media-artifact-alignment
Some checks are pending
Build and Push Docker Images / create_manifest (web, surfsense-web, , cpu) (push) Blocked by required conditions
Build and Push Docker Images / create_manifest (backend, surfsense-backend, -cuda126, cuda126) (push) Blocked by required conditions
Build and Push Docker Images / create_manifest (sandbox, surfsense-sandbox, , cpu) (push) Blocked by required conditions
Build and Push Docker Images / compute_version (push) Waiting to run
Build and Push Docker Images / build (./docker/sandbox, cpu, ./docker/sandbox/Dockerfile, sandbox, surfsense-sandbox, ubuntu-24.04-arm, linux/arm64, arm64, , false, cpu) (push) Blocked by required conditions
Build and Push Docker Images / build (./docker/sandbox, cpu, ./docker/sandbox/Dockerfile, sandbox, surfsense-sandbox, ubuntu-latest, linux/amd64, amd64, , false, cpu) (push) Blocked by required conditions
Build and Push Docker Images / build (./surfsense_backend, cpu, ./surfsense_backend/Dockerfile, backend, surfsense-backend, ubuntu-24.04-arm, linux/arm64, arm64, , production, false, cpu) (push) Blocked by required conditions
Build and Push Docker Images / build (./surfsense_backend, cpu, ./surfsense_backend/Dockerfile, backend, surfsense-backend, ubuntu-latest, linux/amd64, amd64, , production, false, cpu) (push) Blocked by required conditions
Build and Push Docker Images / build (./surfsense_backend, cu126, ./surfsense_backend/Dockerfile, backend, surfsense-backend, ubuntu-24.04-arm, linux/arm64, arm64, -cuda126, production, true, cuda126) (push) Blocked by required conditions
Build and Push Docker Images / build (./surfsense_backend, cu126, ./surfsense_backend/Dockerfile, backend, surfsense-backend, ubuntu-latest, linux/amd64, amd64, -cuda126, production, true, cuda126) (push) Blocked by required conditions
Build and Push Docker Images / build (./surfsense_backend, cu128, ./surfsense_backend/Dockerfile, backend, surfsense-backend, ubuntu-24.04-arm, linux/arm64, arm64, -cuda, production, true, cuda) (push) Blocked by required conditions
Build and Push Docker Images / build (./surfsense_backend, cu128, ./surfsense_backend/Dockerfile, backend, surfsense-backend, ubuntu-latest, linux/amd64, amd64, -cuda, production, true, cuda) (push) Blocked by required conditions
Build and Push Docker Images / build (./surfsense_web, cpu, ./surfsense_web/Dockerfile, web, surfsense-web, ubuntu-24.04-arm, linux/arm64, arm64, , runner, false, cpu) (push) Blocked by required conditions
Build and Push Docker Images / build (./surfsense_web, cpu, ./surfsense_web/Dockerfile, web, surfsense-web, ubuntu-latest, linux/amd64, amd64, , runner, false, cpu) (push) Blocked by required conditions
Build and Push Docker Images / verify_digests (push) Blocked by required conditions
Build and Push Docker Images / create_manifest (backend, surfsense-backend, , cpu) (push) Blocked by required conditions
Build and Push Docker Images / create_manifest (backend, surfsense-backend, -cuda, cuda) (push) Blocked by required conditions
Build and Push Docker Images / finalize_release (push) Blocked by required conditions
[Feat] Media deliverables are Artifacts (image cutover)
2026-08-13 16:17:14 +02:00
CREDO23
29c9333d76 fix(podcast): import the shared query client in the player
The module exports a `queryClient` singleton from `query-client/client`;
the missing `getQueryClient` barrel broke the dev server build.
2026-08-13 15:54:10 +02:00
Anish Sarkar
d15cf094c9 Implement folder deletion confirmation dialog in DocumentRightPanel
- Added state management for pending folder deletion and deletion status.
- Refactored folder deletion logic to include a confirmation dialog before proceeding with the deletion.
- Integrated alert dialog to notify users of the deletion action and its consequences.
2026-08-13 18:57:20 +05:30
CREDO23
499aa4d14d feat(artifacts): deliver generated images straight to an Artifact
generate_image now bills, calls the provider once and persists through
save_artifact, with the generation parameters and revised prompt kept as
provenance. The ImageGeneration model, its routes, schemas, RBAC permissions
and per-image access tokens are gone, along with the frontend service and
types; viewers fetch the primary file through the artifact manifest.

Shared chats keep working: the snapshot stores the artifact ids its messages
reference and a public route serves only those, so a token grants the thread's
own artifacts rather than the workspace's.
2026-08-13 14:58:06 +02:00
Anish Sarkar
d1164e6d97 Enhance Docker setup for development environment
- Updated docker-compose.dev.yml to include a persistent pnpm store for caching dependencies.
- Modified Dockerfile to reconcile node_modules with the lockfile before starting the development server, ensuring consistent dependency management.
2026-08-13 18:15:38 +05:30
CREDO23
fa8066a61c chore(artifacts): drop migration narration from media comments
Comments described the cutover phases rather than any constraint the code
can't show. Keep only the conditions a reader needs (public shares, media
with no Artifact row) and let the slide-key filter speak for itself.
2026-08-13 09:14:48 +02:00
CREDO23
0ce4cae621 feat(artifacts): serve Remotion video payload from Artifact metadata
Store slides/scene_codes on dual-write; expose ARTIFACTS_READ video + slide
audio routes; library opens Remotion via artifactId.
2026-08-13 09:08:33 +02:00
CREDO23
ee99110a4c merge(upstream/dev): DocumentType.ARTIFACT into media alignment
Keep mate's document-sidecar Artifact foundation; retain media dual-write
and Phase 3 legacy list/manifest bridges via persist_artifact shim.
2026-08-13 09:02:58 +02:00
CREDO23
c60a013ca6 feat(artifacts): prefer artifact_id in chat rail and image tool
Split artifactId vs legacyEntityId in the chat collector, return artifact_id
from generate_image, and align roster tests with the current filename shape.
2026-08-13 08:16:23 +02:00
CREDO23
238abb5889 feat(artifacts): surface artifact_id on podcast and video detail
Reverse-lookup dual-written Artifacts after READY so chat cards and clients
can stream via Artifact without waiting on tool-create payloads.
2026-08-13 08:16:23 +02:00
CREDO23
341844f0bd feat(artifacts): stream image and podcast via Artifact files
Library viewers fetch primary bytes under ARTIFACTS_READ. Video still opens
Remotion through metadata.legacy until slides live on Artifact.
2026-08-13 08:16:23 +02:00
CREDO23
8c287ea81c feat(artifacts): prefer Artifact rows in the library list
Artifact list is primary for files and dual-written media; legacy podcast/
video/image endpoints only fill gaps until Phase 4 backfill.
2026-08-13 08:16:23 +02:00
CREDO23
f3c36a5d59 feat(artifacts): expose legacy refs on list and manifest
Phase 3 cutover: dual-written media need metadata.legacy so the library can
dedupe and open Remotion via the old presentation id.
2026-08-13 08:16:12 +02:00
Thierry CH.
e90ceba275
Merge pull request #1676 from CREDO23/fix/hitl-parent-interrupt-routing
Some checks are pending
Build and Push Docker Images / build (./surfsense_backend, cu128, ./surfsense_backend/Dockerfile, backend, surfsense-backend, ubuntu-latest, linux/amd64, amd64, -cuda, production, true, cuda) (push) Blocked by required conditions
Build and Push Docker Images / build (./surfsense_web, cpu, ./surfsense_web/Dockerfile, web, surfsense-web, ubuntu-24.04-arm, linux/arm64, arm64, , runner, false, cpu) (push) Blocked by required conditions
Build and Push Docker Images / build (./surfsense_web, cpu, ./surfsense_web/Dockerfile, web, surfsense-web, ubuntu-latest, linux/amd64, amd64, , runner, false, cpu) (push) Blocked by required conditions
Build and Push Docker Images / compute_version (push) Waiting to run
Build and Push Docker Images / build (./docker/sandbox, cpu, ./docker/sandbox/Dockerfile, sandbox, surfsense-sandbox, ubuntu-24.04-arm, linux/arm64, arm64, , false, cpu) (push) Blocked by required conditions
Build and Push Docker Images / build (./docker/sandbox, cpu, ./docker/sandbox/Dockerfile, sandbox, surfsense-sandbox, ubuntu-latest, linux/amd64, amd64, , false, cpu) (push) Blocked by required conditions
Build and Push Docker Images / build (./surfsense_backend, cpu, ./surfsense_backend/Dockerfile, backend, surfsense-backend, ubuntu-24.04-arm, linux/arm64, arm64, , production, false, cpu) (push) Blocked by required conditions
Build and Push Docker Images / build (./surfsense_backend, cpu, ./surfsense_backend/Dockerfile, backend, surfsense-backend, ubuntu-latest, linux/amd64, amd64, , production, false, cpu) (push) Blocked by required conditions
Build and Push Docker Images / build (./surfsense_backend, cu126, ./surfsense_backend/Dockerfile, backend, surfsense-backend, ubuntu-24.04-arm, linux/arm64, arm64, -cuda126, production, true, cuda126) (push) Blocked by required conditions
Build and Push Docker Images / build (./surfsense_backend, cu126, ./surfsense_backend/Dockerfile, backend, surfsense-backend, ubuntu-latest, linux/amd64, amd64, -cuda126, production, true, cuda126) (push) Blocked by required conditions
Build and Push Docker Images / verify_digests (push) Blocked by required conditions
Build and Push Docker Images / create_manifest (backend, surfsense-backend, , cpu) (push) Blocked by required conditions
Build and Push Docker Images / create_manifest (backend, surfsense-backend, -cuda, cuda) (push) Blocked by required conditions
Build and Push Docker Images / create_manifest (backend, surfsense-backend, -cuda126, cuda126) (push) Blocked by required conditions
Build and Push Docker Images / create_manifest (sandbox, surfsense-sandbox, , cpu) (push) Blocked by required conditions
Build and Push Docker Images / create_manifest (web, surfsense-web, , cpu) (push) Blocked by required conditions
Build and Push Docker Images / finalize_release (push) Blocked by required conditions
Build and Push Docker Images / build (./surfsense_backend, cu128, ./surfsense_backend/Dockerfile, backend, surfsense-backend, ubuntu-24.04-arm, linux/arm64, arm64, -cuda, production, true, cuda) (push) Blocked by required conditions
[Fix] Multi-agent HITL reliability + OpenRouter truncation handling
2026-08-13 08:07:42 +02:00
CREDO23
a8f819a09c merge(upstream/dev): bring DocumentType.ARTIFACT foundation into sheet branch 2026-08-13 07:41:57 +02:00
Anish Sarkar
5f45e31719 feat(artifacts): enhance artifact schemas with document ID and status 2026-08-13 03:55:01 +05:30
Anish Sarkar
81cd76e86d feat(citations): route artifact chunks through document lookup 2026-08-13 03:54:30 +05:30
Anish Sarkar
716fc6615c refactor(citations): remove artifact chunk token kind 2026-08-13 03:54:27 +05:30
Anish Sarkar
4c8806b264 feat(contracts): add ARTIFACT document type 2026-08-13 03:54:22 +05:30
CREDO23
ee8775ce86 test(artifacts): move parseWorkbook tests to tests/unit 2026-08-12 14:42:40 +02:00
CREDO23
ac0d9c78b5 fix(artifacts): auth XlsxViewer file fetch 2026-08-12 14:42:40 +02:00
CREDO23
ef59c5ddd2 feat(artifacts): add XlsxViewer 2026-08-12 13:56:47 +02:00
CREDO23
5276e42b0f test(artifacts): unit-test parseWorkbook 2026-08-12 13:56:47 +02:00
CREDO23
52e90f09a4 feat(artifacts): add parseWorkbook helper 2026-08-12 13:56:47 +02:00
CREDO23
797d9a7311 chore(web): add exceljs and ssf 2026-08-12 13:56:47 +02:00
Anish Sarkar
054b3f9b35 fix(pdf-viewer): reset scroll position to top after rendering new pages 2026-08-12 16:33:18 +05:30
Anish Sarkar
a4a332d80c style(pdf-viewer): adjust padding and margins for improved layout 2026-08-12 16:13:17 +05:30
Anish Sarkar
d8ece9e11c fix(pdf-viewer): update pdfjs-dist versioning and enhance viewer styles 2026-08-12 16:03:07 +05:30
Anish Sarkar
9ab5bd530b feat(artifacts): introduce mobile artifact drawer and refactor artifact panel to viewer 2026-08-12 15:19:52 +05:30