feat(core): add PDF vision bridge fallback (#6846)

* feat(core): add PDF vision bridge fallback Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com> * codex: address PR review feedback (#6846) Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com> * codex: address PR review feedback (#6846) Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com> * codex: fix CI failure on PR #6846 Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com> * codex: address PR review feedback (#6846) Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com> * codex: address PR review feedback (#6846) Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com> * fix(cli): harden vision bridge output handling Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com> * fix(cli): correct export sanitizer test typing Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com> * fix(core): disclose selected vision endpoint before egress Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com> --------- Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
2026-08-10 17:27:10 +00:00 · 2026-07-15 11:42:14 +08:00 · 2026-07-15 11:42:14 +08:00 · 4f4387cf57
commit 4f4387cf57
parent 4b802ca5f9
27 changed files with 2130 additions and 279 deletions
--- a/docs/design/2026-07-13-pdf-vision-bridge-fallback.md
+++ b/docs/design/2026-07-13-pdf-vision-bridge-fallback.md
@ -0,0 +1,25 @@
+# PDF vision bridge fallback
+
+## Context
+
+`read_file` is text-first for PDFs when the primary model lacks native PDF support. Text extraction can still fail for scanned documents, and a single dense page can exceed the safe 12K-token tool-result budget. Returning rendered pages directly is not safe for a text-only provider, while treating every large text result as an image would make ordinary multi-page reads slower and less precise.
+
+## Design
+
+The file-processing layer can prepare an internal, PDF-only vision bridge candidate. This option is separate from the existing unsupported-image preservation used by interactive `@` attachments, so ordinary image reads do not change. A candidate contains rendered image parts, the trigger reason, the actual rendered page range, structured continuation metadata, and the original text-extraction error to restore if transcription cannot complete. Continuation metadata distinguishes pages known to exist from pages that may exist when page counting is unavailable.
+
+Candidates are created only when PDF text extraction fails or when an explicit or actual single-page read still exceeds 12K estimated tokens. Multi-page text overflow, large-document page-range gates, and file-size gates retain their existing guidance. Rendering starts at the requested first page and processes at most four pages per `read_file` call. The requested range is clipped to the PDF's actual page count when known: a six-page document requested as `pages: "4-8"` renders pages 4-6 and does not invent pages 7-8. When page counting is unavailable, a short, non-byte-truncated render is treated as end-of-file; a full four-page render or byte truncation reports only that additional requested pages may exist.
+
+`ReadFileTool` enables preparation only when the primary model is text-only and a vision bridge model is configured or available. It invokes the bridge before building the final tool response, passing only the rendered image pages plus structured PDF page context. The bridge is instructed to label transcription sections with original PDF page numbers. Continuation guidance is appended after transcription and points only to the original PDF, never to temporary rendered images.
+
+On success, `read_file` returns untrusted, lossy machine transcription and no image data. A structured display notice discloses the selected vision model, endpoint when known, transcribed page range, and known or possible continuation. The TUI renders this notice even when successful read output is collapsed and when transcript detail is expanded; ACP, non-interactive structured output, and session exports include the same text in tool-call content rather than relying on opaque raw output. On bridge failure, empty output, timeout, or model-selection changes, the image data is discarded and the exact original PDF error is restored to the model while the bridge attempt remains visible only in the user display. User cancellation propagates. Consequently, no candidate image can reach a text-only primary provider through a tool result.
+
+An explicitly configured `visionModel` is treated as authorization to use that model even when it is hosted by another provider. The existing bridge notice reports the actual endpoint so the data boundary remains visible.
+
+## Compatibility
+
+The public `read_file` schema is unchanged. Native PDF models, vision-capable primary models, configurations without a bridge model, ordinary PNG/JPEG reads, and existing interactive image behavior retain their current paths. Interactive `@` PDF resolution additionally benefits from the single-page overflow fallback.
+
+## Verification
+
+Unit coverage exercises requested ranges that do not begin at page 1, requests extending past the actual document end, unknown page counts, byte truncation, empty renders, single- versus multi-page overflow, bridge success and failures, cancellation, configuration changes, endpoint disclosure across TUI/ACP/export surfaces, page-number prompts, and the invariant that text-only results contain no `inlineData`. E2E verification compares the global baseline with the local build using a six-page scanned PDF, a dense single-page PDF, and a multi-page text-heavy PDF.
--- a/docs/developers/tools/file-system.md
+++ b/docs/developers/tools/file-system.md
@ -24,18 +24,21 @@ Qwen Code provides a comprehensive suite of tools for interacting with the local

 ## 2. `read_file` (ReadFile)

-`read_file` reads and returns the content of a specified file. This tool handles text files and media files (images, PDFs, audio, video) whose modality is supported by the current model. For text files, it can read specific line ranges. Media files whose modality is not supported by the current model are rejected with a helpful error message. Other binary file types are generally skipped.
+`read_file` reads and returns the content of a specified file. This tool handles text files and media files (images, PDFs, audio, video) whose modality is supported by the current model. For text files, it can read specific line ranges. Unsupported PDFs attempt text extraction and the bounded vision fallback described below; other unsupported media files return a helpful error message. Other binary file types are generally skipped.

 - **Tool name:** `read_file`
 - **Display name:** ReadFile
 - **File:** `read-file.ts`
 - **Parameters:**
-  - `path` (string, required): The absolute path to the file to read.
+  - `file_path` (string, required): The absolute path to the file to read.
  - `offset` (number, optional): For text files, the 0-based line number to start reading from. Requires `limit` to be set.
  - `limit` (number, optional): For text files, the maximum number of lines to read. If omitted, reads a default maximum (e.g., 2000 lines) or the entire file if feasible.
+  - `pages` (string, optional): For PDFs, a 1-indexed page or closed page range such as `"3"` or `"20-25"`. A request may contain at most 20 pages.
 - **Behavior:**
  - For text files: Returns the content. If `offset` and `limit` are used, returns only that slice of lines. Indicates if content was truncated due to line limits or line length limits.
  - For media files (images, PDFs, audio, video): If the current model supports the file's modality, returns the file content as a base64-encoded `inlineData` object. If the model does not support the modality, returns an error message with guidance (e.g., suggesting skills or external tools).
+  - For PDFs with a text-only primary model: Text extraction is attempted first. If extraction fails, or an explicitly requested (or actual) single page still exceeds the 12K-token text budget, a configured vision bridge automatically renders and transcribes at most four pages beginning at the requested first page. The requested range is clipped to the actual document end when known. The result identifies the transcribed range and either the pages known to remain or, when the page count is unavailable, that additional pages may exist. Ordinary multi-page text overflow still asks for a narrower `pages` range instead of switching to vision.
+  - Vision bridge PDF transcription is lossy and marked as untrusted machine-generated content. The tool result contains text rather than rendered images, and its user-facing TUI, ACP, non-interactive structured output, and export displays identify the vision model and endpoint when known. If the bridge fails, the exact original PDF extraction error is returned to the model while the user display still discloses the bridge attempt.
  - For other binary files: Attempts to identify and skip them, returning a message indicating it's a generic binary file.
 - **Output:** (`llmContent`):
  - For text files: The file content, potentially prefixed with a truncation message (e.g., `[File content truncated: showing lines 1-100 of 500 total lines...]\nActual file content...`).
--- a/docs/users/configuration/settings.md
+++ b/docs/users/configuration/settings.md
@ -258,9 +258,9 @@ The `extra_body` field allows you to add custom parameters to the request body s

 #### visionModel

-| Setting       | Type   | Description                                                                                                                                                                                                                        | Default |
-| ------------- | ------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------- |
-| `visionModel` | string | Image-capable model used as the vision bridge: when a text-only main model receives an image, it is transcribed by this model first. Leave empty to auto-pick a same-provider vision model. Can also be set via `/model --vision`. | `""`    |
+| Setting       | Type   | Description                                                                                                                                                                                                                                                                                                                                                                                                                         | Default |
+| ------------- | ------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------- |
+| `visionModel` | string | Image-capable model used as the vision bridge: when a text-only main model receives an image, or `read_file` needs the bounded PDF visual fallback, it is transcribed by this model first. Setting this explicitly authorizes bridge calls to that model even when it uses another provider; the tool display discloses the endpoint. Leave empty to auto-pick a same-provider vision model. Can also be set via `/model --vision`. | `""`    |

 #### visionBridgeTimeoutMs