Commit graph

41 commits

Author SHA1 Message Date
Timothy Jaeryang Baek
f1409266fe refac 2026-07-27 02:44:21 -04:00
G30
771540f3de
feat: sortable title and updated at columns on the knowledge workspace page (#27457) 2026-07-26 18:19:31 -04:00
Classic298
310ae91302
perf: make tools defer_content real and batch tool and knowledge access filters (#27387)
Some checks are pending
Python CI / Ruff Format (3.11) (push) Waiting to run
Python CI / Ruff Format (3.12) (push) Waiting to run
Create and publish Docker images with specific build args / build (map[arch:linux/amd64 runner:ubuntu-latest], map[build_args:USE_OLLAMA=true free_disk:false name:ollama suffix:-ollama]) (push) Waiting to run
Create and publish Docker images with specific build args / build (map[arch:linux/amd64 runner:ubuntu-latest], map[build_args:USE_SLIM=true free_disk:false name:slim suffix:-slim]) (push) Waiting to run
Create and publish Docker images with specific build args / build (map[arch:linux/arm64 runner:ubuntu-24.04-arm], map[build_args: free_disk:false name:main suffix:]) (push) Waiting to run
Create and publish Docker images with specific build args / build (map[arch:linux/arm64 runner:ubuntu-24.04-arm], map[build_args:USE_CUDA=true USE_CUDA_VER=cu126 free_disk:true name:cuda126 suffix:-cuda126]) (push) Waiting to run
Create and publish Docker images with specific build args / build (map[arch:linux/arm64 runner:ubuntu-24.04-arm], map[build_args:USE_CUDA=true free_disk:true name:cuda suffix:-cuda]) (push) Waiting to run
Create and publish Docker images with specific build args / build (map[arch:linux/arm64 runner:ubuntu-24.04-arm], map[build_args:USE_OLLAMA=true free_disk:false name:ollama suffix:-ollama]) (push) Waiting to run
Create and publish Docker images with specific build args / build (map[arch:linux/arm64 runner:ubuntu-24.04-arm], map[build_args:USE_SLIM=true free_disk:false name:slim suffix:-slim]) (push) Waiting to run
Create and publish Docker images with specific build args / copy-to-dockerhub (-ollama, ollama) (push) Blocked by required conditions
Create and publish Docker images with specific build args / build (map[arch:linux/amd64 runner:ubuntu-latest], map[build_args: free_disk:false name:main suffix:]) (push) Waiting to run
Create and publish Docker images with specific build args / build (map[arch:linux/amd64 runner:ubuntu-latest], map[build_args:USE_CUDA=true USE_CUDA_VER=cu126 free_disk:true name:cuda126 suffix:-cuda126]) (push) Waiting to run
Create and publish Docker images with specific build args / build (map[arch:linux/amd64 runner:ubuntu-latest], map[build_args:USE_CUDA=true free_disk:true name:cuda suffix:-cuda]) (push) Waiting to run
Create and publish Docker images with specific build args / merge (map[name:cuda suffix:-cuda]) (push) Blocked by required conditions
Create and publish Docker images with specific build args / merge (map[name:cuda126 suffix:-cuda126]) (push) Blocked by required conditions
Create and publish Docker images with specific build args / merge (map[name:main suffix:]) (push) Blocked by required conditions
Create and publish Docker images with specific build args / merge (map[name:ollama suffix:-ollama]) (push) Blocked by required conditions
Create and publish Docker images with specific build args / merge (map[name:slim suffix:-slim]) (push) Blocked by required conditions
Create and publish Docker images with specific build args / notify-helm-charts (push) Blocked by required conditions
Create and publish Docker images with specific build args / copy-to-dockerhub (, main) (push) Blocked by required conditions
Create and publish Docker images with specific build args / copy-to-dockerhub (-cuda, cuda) (push) Blocked by required conditions
Create and publish Docker images with specific build args / copy-to-dockerhub (-cuda126, cuda126) (push) Blocked by required conditions
Create and publish Docker images with specific build args / copy-to-dockerhub (-slim, slim) (push) Blocked by required conditions
Frontend Build / Format & Build (push) Waiting to run
Frontend Build / Unit Tests (push) Waiting to run
Tools.get_tools(defer_content=True) contained the literal dead statement "stmt = stmt": the deferral was a no-op, so every tools listing loaded the full Python source of every tool (five caller sites pass defer_content=True expecting the optimization: the tools list endpoints and the user and group permission overviews). The listing now selects every column except content, and ToolModel.content becomes optional to represent deferred rows; router projections are content-less response models, so nothing downstream reads the source on these paths.

On top of that, get_tools_by_user_id issued one grant query per non-owned tool and Knowledges.get_knowledge_bases_by_user_id did the same per knowledge base (the latter also sits inside per-file access checks). Both now resolve grants for all non-owned rows in a single get_accessible_resource_ids call, the same batch helper the model listing already uses.

Benchmark (real SQLite DB):

| metric | before | after |
| --- | --- | --- |
| tools listing, 33 tools x ~200 KB source | 5.13 ms | 3.18 ms |
| grant queries per accessible-tools call, N non-owned tools | N | 1 |
| grant queries per accessible-KBs call, N non-owned KBs | N | 1 |

The listing row scales with source size; on Postgres the deferral additionally avoids shipping every tool's source over the wire per listing, and each removed grant query was a real round trip.

Functionally verified: deferred listings match full listings field for field with content None, grants included and router projections working; access filtering returns exactly owned plus granted tools and knowledge bases and nothing for strangers; full (non-deferred) reads still carry the source.
2026-07-23 17:51:07 -05:00
Classic298
5b518cbe43
perf: list knowledge base file metadata without loading extracted file contents (#27386)
Knowledges.get_file_metadatas_by_id fetched full File rows, whose data column carries the entire extracted text of each document, validated each into a FileModel and then threw everything except id, hash, meta and the timestamps away. The function backs every knowledge base detail view and runs again after every file add or remove (eight call sites in the knowledge router), so rendering a filename list for a 50-file knowledge base parsed tens of megabytes of JSON per request.

The listing now selects exactly the five columns the response needs, joined through KnowledgeFile, mirroring the column-only helper that already existed in the files model for id-based lookups.

Benchmark (real SQLite DB, 50 files with ~200 KB extracted text each):

| metric | before | after |
| --- | --- | --- |
| KB file metadata listing | 14.2 ms | 1.20 ms |

The gap widens linearly with file size and count since extracted contents no longer get read, parsed or validated at all.

Functionally verified: output matches the old implementation field for field on all 50 files and an unknown knowledge base still returns an empty list.
2026-07-23 17:50:55 -05:00
Timothy Jaeryang Baek
e5e2cd7876 refac 2026-07-17 04:48:21 -04:00
Timothy Jaeryang Baek
ab84bbf08c refac 2026-06-29 04:19:33 -05:00
Timothy Jaeryang Baek
46c1d6591b refac 2026-06-29 01:41:43 -05:00
Timothy Jaeryang Baek
15c7e37438 refac 2026-06-23 23:13:32 +02:00
Timothy Jaeryang Baek
591e0aafa1 refac 2026-05-28 17:24:33 -05:00
Timothy Jaeryang Baek
1527eb6e01 refac 2026-05-20 01:46:59 +04:00
Timothy Jaeryang Baek
4e78b355ef refac 2026-05-14 02:00:23 +09:00
Timothy Jaeryang Baek
d74ee34d91 refac 2026-05-14 01:49:34 +09:00
Timothy Jaeryang Baek
c2cbc47ca7 feat: knowledge directory 2026-05-13 22:37:53 +09:00
Timothy Jaeryang Baek
6d0295588e refac: modernize type annotations (PEP 604 / PEP 585) 2026-05-12 17:10:15 +09:00
Timothy Jaeryang Baek
11e076817a refac 2026-05-09 07:34:46 +09:00
Timothy Jaeryang Baek
25898116ea chore: format 2026-04-12 18:12:59 -05:00
Timothy Jaeryang Baek
27169124f2 refac: async db 2026-04-12 14:22:11 -05:00
Timothy Jaeryang Baek
ade617efa8 refac 2026-03-24 04:49:48 -05:00
Timothy Jaeryang Baek
de3317e26b refac 2026-03-17 17:58:01 -05:00
Timothy Jaeryang Baek
fcff9c3afd refac 2026-03-01 13:20:55 -06:00
Timothy Jaeryang Baek
631e30e22d refac 2026-02-21 15:35:34 -06:00
Timothy Jaeryang Baek
8c485b260f refac 2026-02-19 16:53:21 -06:00
Timothy Jaeryang Baek
f376d4f378 chore: format 2026-02-11 16:24:11 -06:00
Timothy Jaeryang Baek
0f78451c2b refac 2026-02-08 21:24:20 -06:00
Timothy Jaeryang Baek
a9a0ce6bea refac 2026-01-22 03:09:04 +04:00
G30
e9926694c3
fix: add username search support to workspace and admin pages (#20780)
This fix restores and extends the username/email search functionality across workspace pages that was originally added in PR #14002. The issue was that:

1. The backend search functions for Models and Knowledge only searched `User.name` and `User.email`, but not `User.username`

2. The Functions admin page lacked user search entirely

Changes made:

Added User.username to backend search conditions for Models and Knowledge pages
Added complete user search (name, email, username) to the Functions admin page client-side filter
2026-01-19 13:42:33 +04:00
Timothy Jaeryang Baek
700349064d chore: format 2026-01-08 01:55:56 +04:00
G30
8712446cc1
fix: resolve undefined knowledge base name in file reference tooltip (#20333) 2026-01-03 18:13:19 +04:00
Timothy Jaeryang Baek
2041ab483e refac/enh: db session sharing 2025-12-28 22:00:44 +04:00
Classic298
823b9a6dd9
chore/perf: Remove old SRC level log env vars with no impact (#20045)
* Update openai.py

* Update env.py

* Merge pull request open-webui#19030 from open-webui/dev (#119)

Co-authored-by: Tim Baek <tim@openwebui.com>
Co-authored-by: Claude <noreply@anthropic.com>

---------

Co-authored-by: Tim Baek <tim@openwebui.com>
Co-authored-by: Claude <noreply@anthropic.com>
2025-12-20 08:16:14 -05:00
Timothy Jaeryang Baek
4d9a51ba33 refac 2025-12-11 00:11:12 -05:00
Timothy Jaeryang Baek
ceae3d48e6 enh/refac: kb pagination 2025-12-10 23:19:19 -05:00
Timothy Jaeryang Baek
c15201620d refac: kb files 2025-12-10 15:48:27 -05:00
Timothy Jaeryang Baek
94a8439105 feat/enh: kb file pagination 2025-12-10 00:53:41 -05:00
Timothy Jaeryang Baek
e301d1962e refac/perf: has_access_to_file optimization 2025-12-02 11:11:17 -05:00
Timothy Jaeryang Baek
d19023288e feat/enh: kb files db migration 2025-12-02 10:53:32 -05:00
Timothy Jaeryang Baek
40e40d1ddd enh/sec: verify folder data integrity
#17182
2025-09-07 01:04:56 +04:00
Tim Jaeryang Baek
472b71f331
Merge pull request #17166 from sihyeonn/perf/sh-model-layer
perf: fix N+1 query issues in user group access control validation
2025-09-03 13:36:01 +04:00
Sihyeon Jang
eff06538a6 perf: fix N+1 query issues in user group access control validation
- Pre-fetch user group IDs in get_*_by_user_id methods across models layer
- Pass user_group_ids to has_access to avoid repeated group queries
- Reduce query count from 1+N to 1+1 pattern for access control validation
- Apply consistent optimization across knowledge, models, notes, prompts, and tools

Signed-off-by: Sihyeon Jang <sihyeon.jang@navercorp.com>
2025-09-03 05:56:48 +09:00
Sihyeon Jang
f588655f7f perf: fix N+1 query issue in get_knowledge_bases method
- Replace individual user queries with batch fetching
- Use single query to fetch all required users at once
- Implement O(1) user lookup with dictionary mapping
- Reduce query count from 1+N to 1+1 pattern

Signed-off-by: Sihyeon Jang <sihyeon.jang@navercorp.com>
2025-09-03 05:29:47 +09:00
Timothy Jaeryang Baek
d3d161f723 wip 2024-12-10 00:54:13 -08:00
Renamed from backend/open_webui/apps/webui/models/knowledge.py (Browse further)