1 commit

Author	SHA1	Message	Date
Daniel Han	d8effae0d5	Studio: fix the Windows desktop setup log mojibake and double-printed steps (#8083) ... * Studio: fix the Windows desktop setup log mojibake and double-printed steps The desktop 'Getting things ready...' log rendered as: ?? Unsloth Studio Setup <52 replacement chars> gpu none (chat-only / GGUF) gpu none (chat-only / GGUF) Encoding. studio/setup.ps1 never set [Console]::OutputEncoding, so Windows PowerShell 5.1 encoded redirected output with the OEM code page while the desktop app decodes the pipe as UTF-8 (String::from_utf8_lossy in src-tauri/src/install.rs). That corrupts two different ways: the sloth U+1F9A5 has no OEM representation so PowerShell substitutes one '?' per UTF-16 surrogate, and the rule U+2500 does have one, so it becomes a bare 0xC4 byte that is invalid UTF-8 and surfaces as U+FFFD. Both entry scripts now set the console encoding, $OutputEncoding, PYTHONUTF8 and PYTHONIOENCODING before the first write, and Refresh-Environment can no longer reload the two Python vars back over ours mid-run. The patch is ASCII-only: these files are UTF-8 without a BOM and 5.1 parses those as ANSI. Duplication. step/substep wrote through Write-Host AND a console-handle mirror. The mirror's comment assumed Write-Host does not survive the process chain; it does, because the CLI spawns setup.ps1 as -Command "& '...' *>&1" (unsloth_cli/commands/studio.py), which merges the Information stream into stdout deliberately. The sink is now resolved once and exactly one is used: redirected writes to the console handle, interactive writes to Write-Host. Splitting. step composed one logical line from two Write-Host calls using -NoNewline, and a redirected consumer turns each Information record boundary into a line break. Both scripts now emit one composed record; install.ps1 needs this most, having no mirror to fall back on. Rust children on Windows get PYTHONUTF8/PYTHONIOENCODING too, since install.rs, update.rs and process.rs all decode their output as UTF-8. The readers stay lossy on purpose -- strict decoding would turn display corruption into an installation failure. Tests: a Pester suite auto-discovered by the existing pester job, and a pytest byte-level probe that runs real PowerShell in both the -File and -Command launch shapes and asserts on raw bytes. Verified to fail against the unfixed tree (9 Pester and 13 pytest failures) rather than merely passing. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Bind a UTF-8 writer with no console, and pass -X utf8 to the isolated child Two holes in the previous commit, both on the exact path the desktop app takes. [Console]::OutputEncoding P/Invokes SetConsoleOutputCP, which needs a console handle. Under CREATE_NO_WINDOW there is none, so it throws, and it drops the cached writer BEFORE throwing while assigning OutputEncoding only after. Console .Out therefore rebuilt on the old code page. Swallowing the exception was not enough once redirected step/substep use Console.Out as their only sink, so the catch path now binds an explicit UTF-8 StreamWriter over OpenStandardOutput. build_update_command launches Python with -I, which implies -E, so that process ignores every PYTHON* variable and PYTHONUTF8/PYTHONIOENCODING never reached it. Pass -X utf8 as a switch instead. The env vars stay for its descendants. https://docs.python.org/3/using/cmdline.html#cmdoption-I * Bind the UTF-8 writer to stderr as well when there is no console The no-console fallback repaired Console.Out only. Tauri pipes stderr through the same lossy UTF-8 decode (install.rs) and emits it to the same UI log, and InstallFailureContext builds the user-facing failure message from those lines, so a PowerShell error carrying a non-ASCII path still arrived as U+FFFD. install.ps1 also writes its Clear-TauriInstallError markers there. * Tighten the comments added by this PR Comments only, no code change. Verified with the PowerShell AST tokenizer for both .ps1 files and the Pester suite (token streams identical with Comment and NewLine excluded), comment_tools.py for the Python test, and a code-only diff for update.rs. * Update the Windows command assertion for the added UTF-8 flags windows_update_command_uses_python_not_replaceable_console_stub asserts the exact argument vector, so adding -X utf8 broke it. The Windows cargo test job in studio-tauri-smoke.yml runs it; the Linux job skips it under cfg(windows), and cargo check type-checks tests without running them, so neither the org Linux run nor the staging check caught it. --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>	2026-08-07 01:18:35 -07:00