Add documentation for encoding detection, default encoding settings,
CRLF handling for batch files, and UTF-8 BOM for PowerShell scripts on
Windows.
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
- Add needsUtf8Bom() to detect when UTF-8 BOM is needed based on file
extension and system code page
- PowerShell 5.1 on non-UTF-8 Windows systems (e.g. GBK) requires BOM
to read scripts correctly
- Remove default UTF8 encoding; undefined now triggers auto-detection
- Add tests for needsUtf8Bom() covering Windows/non-Windows scenarios
This ensures PowerShell scripts are written with UTF-8 BOM on systems
that need it, fixing character encoding issues for non-ASCII content.
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
Refactor the vi.mock for 'os' to use a simpler direct mock object
instead of the importOriginal pattern, making the test setup more concise.
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
- Make CRLF conversion for .bat/.cmd files Windows-only
- Extract PowerShell UTF-8 prefix into reusable function
- Replace custom UTF-8 validation with Node.js built-in isUtf8()
This ensures .bat/.cmd files are only converted on Windows where cmd.exe
actually requires CRLF, and reduces code duplication for shell encoding.
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
This removes the Windows-specific encoding integration tests that are
no longer needed after consolidating the encoding utilities.
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
Update JSDoc to make clear that detectEncodingFromBuffer only performs
chardet statistical detection and returns null on failure. Callers like
getCachedEncodingForBuffer are responsible for providing fallback logic.
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
Move system encoding fallback from detectEncodingFromBuffer into
getCachedEncodingForBuffer for clearer responsibility. Remove unused
WINDOWS_UTF8_CODE_PAGE export and inline the value.
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
Reduce test count from 8 to 7, remove script dependencies, and use
system-created test files instead. Tests now focus on real-world
scenarios: GBK dir listings, file content, PowerShell UTF-8 prefix,
and late CJK detection after ASCII output.
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
- Prefix PowerShell commands with [Console]::OutputEncoding=UTF8
- Re-detect encoding on full buffer after streaming completes
- Move system encoding fallback into detectEncodingFromBuffer
This ensures CJK and other non-ASCII characters are correctly decoded
on Windows systems with non-UTF-8 system codepages (e.g., GBK/CP936).
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
Add comprehensive test plan and scripts for verifying encoding detection
on Windows systems with non-UTF-8 codepages (e.g., GBK/CP936).
This provides manual test cases for output decoding, file round-trips,
PTY behavior, and edge cases like large output with late CJK content.
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
- Rename outputEncoding to detectedEncoding for clarity
- Add isValidUtf8 helper using TextDecoder in fatal mode
- Restructure detection: UTF-8 → chardet → system encoding
- Update tests to use non-UTF-8 bytes for accurate testing
This prevents misclassifying UTF-8 output as legacy codepages on systems
where the system encoding (e.g., GBK) could also decode those bytes.
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
Remove wrapCommandForWindowsEncoding and forceUtf8Output parameter,
relying solely on getCachedEncodingForBuffer for encoding detection.
This simplifies the shell execution flow by removing the chcp 65001
command prefixing approach.
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
- Add automatic LF to CRLF conversion when writing .bat/.cmd files
- Handle null encoding detection gracefully in shellExecutionService
- Add comprehensive tests for CRLF conversion edge cases
This ensures Windows batch files work correctly, as cmd.exe requires
CRLF line endings for proper parsing of multi-line constructs.
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
Return forceUtf8Output flag from wrapCommandForWindowsEncoding to
signal that output should be decoded as UTF-8 when we've switched
the codepage to 65001. This ensures consistent encoding for both
command input and output on Windows with non-UTF-8 system codepage.
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
Simplify wrapCommandForWindowsEncoding to always prefix with chcp 65001
on Windows with non-UTF-8 codepage, rather than only for non-ASCII
commands. This ensures script files written by qwen-code are also
interpreted correctly by cmd.exe.
Add WINDOWS_UTF8_CODE_PAGE constant and update test script for
PowerShell 5.1 compatibility.
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
Add wrapCommandForWindowsEncoding() to prefix non-ASCII commands with
chcp 65001 on Windows with non-UTF-8 codepages. Fix CP936 mapping from
gb2312 to gbk (GBK is the correct superset). Update Windows encoding
test scripts to demonstrate the fix.
This ensures Chinese and other non-ASCII characters display correctly
when running commands on Windows systems with GBK/CP936 codepage.
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
- Add Python scripts for generating large text output (2000 lines)
- Add Chinese progress bar shell script for testing terminal output
- Add Windows GBK encoding reproduction tests for qwen-code
These utilities help debug shell output handling and Windows encoding issues.
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
When a file is skipped because the model doesn't support a modality,
it should not be treated as an error. The error field was incorrectly
being set alongside the informational message.
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
Adds the required getContentGeneratorConfig mock to read-file.test.ts
and pathReader.test.ts to fix failing tests that depend on content
generator configuration.
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
This fixes session corruption issues where the modality check was based on
the model name rather than the actual resolved config, causing inconsistent
behavior when the config's modalities differed from the defaults.
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
- Increase description warning threshold from 500 to 1,000 characters
- Change system prompt 10,000 char limit from error to warning
- Remove intermediate 5,000 char warning threshold for system prompts
- Update documentation to reflect soft warning behavior
This provides more flexibility for users while still guiding them
toward better practices.
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
Restructures the convertOpenAIResponseToGemini method to set
response.candidates within the choice conditional block, making
the empty choices handling more explicit and the control flow clearer.
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
- Add 'array' type support to SettingItemDefinition
- Change hooks field from object to array type
- Add additionalProperties constraint for env fields
- Fix additionalProperties generation to only apply for object types
This ensures the hooks configuration schema correctly represents hooks as an array
and properly validates environment variable objects.
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
Replay raw PTY output through a terminal buffer to properly collapse
carriage-return progress updates (e.g., git clone progress) instead of
showing all intermediate states.
This ensures the final output reflects what a user would actually see
in their terminal, with progress bars and spinners collapsed to their
final state.
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
The Terminal.prototype.write mock's data parameter needed to accept
string | Uint8Array to match the actual method signature.
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
StartSessionEvent requires getTruncateToolOutputThreshold,
getTruncateToolOutputLines, getIdeMode, getShouldUseNodePtyShell,
and getHookSystem from the config object.
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
When PTY outputs data rapidly, the headless terminal rendering is
slower than appending buffers, and the render queue can be overrun
before finalize() races on exit. Additionally, node-pty can fire
onExit before all onData events have been delivered.
This fix:
- Captures raw output immediately before processing
- Uses setImmediate drain mechanism to flush late onData events
- Builds output from raw chunks instead of xterm buffer to avoid
scrollback limit data loss
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
