mirror of
https://github.com/bytedance/deer-flow.git
synced 2026-08-17 12:23:38 +00:00
docs: record audited upload adapters
This commit is contained in:
parent
d170c7e088
commit
1eed820213
2 changed files with 19 additions and 11 deletions
|
|
@ -4,7 +4,7 @@
|
|||
|
||||
**Goal:** Make every DeerFlow upload ingress preserve same-name files under concurrent writes while isolating generated Markdown so conversion and deletion never overwrite or remove user uploads.
|
||||
|
||||
**Architecture:** A shared upload manager stages complete payloads and publishes them with an atomic hard-link no-replace primitive, returning the actual `_N` filename. A focused layout module maps each primary upload to a system-owned `.upload-conversions/<full-primary-name>.md` asset, and a conversion wrapper publishes that asset atomically. Gateway, embedded client, generic IM, and DingTalk become transport adapters over these shared primitives.
|
||||
**Architecture:** A shared upload manager stages complete payloads and publishes them with an atomic hard-link no-replace primitive, returning the actual `_N` filename. A focused layout module maps each primary upload to a system-owned `.upload-conversions/<full-primary-name>.md` asset, and a conversion wrapper publishes that asset atomically. Gateway, embedded client, generic IM, Feishu, DingTalk, and WeChat download staging become transport adapters over these shared primitives.
|
||||
|
||||
**Tech Stack:** Python 3.12+, pathlib/os/tempfile/shutil, asyncio, FastAPI/Pydantic, pytest/pytest-asyncio, unittest.mock, Ruff.
|
||||
|
||||
|
|
@ -1135,11 +1135,11 @@ In DingTalk `_persist`, keep owner-scoped directory creation but replace the loc
|
|||
paths.ensure_thread_dirs(thread_id, user_id=effective_user_id)
|
||||
uploads_dir = paths.sandbox_uploads_dir(
|
||||
thread_id, user_id=effective_user_id
|
||||
).resolve()
|
||||
)
|
||||
return publish_upload_bytes(uploads_dir, safe_filename, content)
|
||||
```
|
||||
|
||||
Remove `_file_write_lock` because the shared primitive supplies cross-thread and cross-process correctness. Keep the other DingTalk locks. Build the returned path through `upload_virtual_path(resolved_target.name)`.
|
||||
Do not resolve the upload directory before publication: the shared publisher must inspect and reject a planted directory symlink itself. Remove `_file_write_lock` because the shared primitive supplies cross-thread and cross-process correctness. Keep the other DingTalk locks. Build the returned path through `upload_virtual_path(resolved_target.name)`.
|
||||
|
||||
- [ ] **Step 5: Refresh the channel blocking-I/O regression descriptions**
|
||||
|
||||
|
|
|
|||
|
|
@ -1,8 +1,8 @@
|
|||
# Upload Collision Safety Design
|
||||
|
||||
**Issue:** #3750
|
||||
**Status:** Approved in conversation; awaiting review of this written specification
|
||||
**Scope:** Gateway uploads, embedded client uploads, inbound IM attachments, DingTalk attachments, generated Markdown conversions, outline lookup, and upload deletion
|
||||
**Issue:** #3750
|
||||
**Status:** Approved
|
||||
**Scope:** Gateway uploads, embedded client uploads, inbound IM attachments, generated Markdown conversions, outline lookup, and upload deletion
|
||||
|
||||
## Problem
|
||||
|
||||
|
|
@ -100,7 +100,8 @@ request-local `seen_filenames` are removed from adapters rather than retained as
|
|||
naming mechanism.
|
||||
|
||||
The publisher returns the actual filename chosen. Gateway, embedded client, generic IM,
|
||||
and DingTalk adapters all use it instead of implementing their own scan-then-write flow.
|
||||
Feishu, DingTalk, and WeChat download staging all use it instead of implementing their
|
||||
own scan-then-write flow.
|
||||
|
||||
## Conversion Publication and Ownership
|
||||
|
||||
|
|
@ -152,10 +153,17 @@ layout as the Gateway. Repeated calls cannot overwrite an earlier upload.
|
|||
Downloaded attachment bytes are staged and published by the shared publisher. Correctness
|
||||
does not depend on a pre-download directory scan, so parallel messages are safe.
|
||||
|
||||
### DingTalk
|
||||
### Feishu and DingTalk
|
||||
|
||||
DingTalk uses the same publisher. Its in-process lock may remain for provider-specific
|
||||
coordination, but it is not relied on for cross-process or cross-adapter safety.
|
||||
Both direct-to-thread adapters use the same publisher and preserve the unresolved upload
|
||||
directory path so the publisher can reject a planted directory symlink. No process-local
|
||||
lock or pre-publication directory scan is relied on for correctness.
|
||||
|
||||
### WeChat
|
||||
|
||||
WeChat first downloads and decrypts inbound media into its channel state directory before
|
||||
the generic IM ingestion step. This temporary materialization also uses the publisher, so
|
||||
parallel messages cannot overwrite one another before the thread upload copy occurs.
|
||||
|
||||
### Sandbox synchronization
|
||||
|
||||
|
|
@ -190,7 +198,7 @@ Tests are written before implementation and cover:
|
|||
deterministic unique names.
|
||||
2. Existing files and symlinks are never followed or replaced.
|
||||
3. Gateway uploads collide safely across separate and concurrent requests.
|
||||
4. Embedded, generic IM, and DingTalk ingress paths follow the same naming behavior.
|
||||
4. Embedded, generic IM, Feishu, DingTalk, and WeChat ingress paths follow the same naming behavior.
|
||||
5. Same-stem files with different extensions receive distinct conversion paths.
|
||||
6. A user-uploaded `report.md` is neither overwritten by converting `report.pdf` nor
|
||||
removed when `report.pdf` is deleted.
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue