fix: preserve original encoding when reading/writing non-UTF-8 files

Fixes #2069

- Add iconv-lite dependency for non-UTF-8 encoding support
- Add iconvHelper.ts as a CJS/ESM compatibility wrapper
- Update readFileWithEncoding() to detect and handle GBK/Big5/Shift_JIS
  using BOM detection -> UTF-8 validation -> chardet -> iconv-lite fallback
- Add detectFileEncoding() to identify file encoding before writes
- Update writeTextFile() to accept encoding option and encode with iconv-lite
- Update WriteFileTool and EditTool to detect and preserve original encoding
- Add tests for GBK read/write, detectFileEncoding, and encoding option
This commit is contained in:
LaZzyMan 2026-03-04 14:52:46 +08:00
parent 407a66c959
commit a5eb1733fa
11 changed files with 375 additions and 32 deletions

View file

@ -42,6 +42,7 @@
"ajv-formats": "^3.0.0",
"async-mutex": "^0.5.0",
"chardet": "^2.1.0",
"iconv-lite": "^0.6.3",
"chokidar": "^4.0.3",
"diff": "^7.0.0",
"dotenv": "^17.1.0",