unsloth/tests/studio_setup_ps1/Studio.Setup.Output.Tests.ps1
Daniel Han 07df95079e
Studio: route every Windows installer line through the UTF-8 stdout sink (#8148)
* Studio: route every Windows setup line through the UTF-8 stdout sink

The desktop setup log rendered "?? Unsloth Studio Setup" over a rule of
replacement characters. Tauri spawns Windows PowerShell 5.1 with
CREATE_NO_WINDOW (install.rs), so the [Console]::OutputEncoding setter
throws and both entry scripts rebind [Console]::Out to a UTF-8 writer.
step/substep already write only through that writer when stdout is
redirected, so they came out right. Every other line did not: Write-Host
is written by 5.1's console host with its own writer on the OEM code
page, and U+1F9A5 has no OEM form while U+2500 becomes a bare 0xC4, which
from_utf8_lossy turns into U+FFFD. The banner and the footer are not
steps, so they kept arriving as mojibake, and install.ps1 had neither the
IsOutputRedirected probe nor a mirror at all.

Add Write-StudioLine above the first write in studio/setup.ps1 and
install.ps1: console handle when redirected, Write-Host when interactive,
since it is the only writer that colorizes. Rewrite 164 call sites in
setup.ps1 and 155 in install.ps1 onto it, including install.ps1's own
step/substep. Write-Host now survives only inside helpers that have
already ruled out the redirected sink, and the launcher script install.ps1
generates keeps its own, since it runs as a separate process.

No behaviour change for an interactive console user: same text, same
colors, same single record per line.

test_windows_setup_output_encoding.py gains byte-level coverage that the
real banner and footer, sliced out of setup.ps1, survive both launch
shapes as valid UTF-8 exactly once, plus a source contract that runs on
Linux and names any file:line that reaches for Write-Host outside the
allow-list. Studio.Setup.Output.Tests.ps1 covers Write-StudioLine in both
modes and pins install.ps1's copy to setup.ps1's.

Harnesses that splice these scripts apart now stub or dot-source
Write-StudioLine: two PowerShell harnesses, one Python harness, and the
VC++ redist leg of studio-windows-inference-smoke.

pytest tests/python tests/test_installer_*.py: 1077 passed (2 pre-existing
sandbox failures unrelated to this change). All 16 tests/studio harnesses
and 57 Pester cases pass. Both scripts parse clean.

* CI: spawn install.ps1 as a child process so its lines reach install.log

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Stub the output sink in the llama.cpp backend PowerShell harness

* Guard the console-less spawn on a Windows runner

The byte-level cases in this file run with a console attached, and a GitHub
runner gives a CREATE_NO_WINDOW child one, so the UTF-8 setter succeeds there
and every version of these scripts emits a clean banner. Those cases cannot
tell this fix from what preceded it.

Add cases that call FreeConsole() in the child first, which is the state
install.rs's own comment assumes CREATE_NO_WINDOW produces. There Write-Host
has no screen buffer to query, throws, and takes the script down: 2 bytes of
stdout and exit 1 rather than the banner. The probe is assembled entirely out
of text sliced from the script under test and spawned with install.rs's own
interpreter, flags and creation flags.

No Windows job ran this file, so its byte-level half was only ever exercised
under pwsh 7 on the Linux Backend CI leg, which is UTF-8 by default. Add it to
the cross-platform parity matrix, which already has a windows-latest row and
already triggers on install.ps1 and studio/setup.ps1.

* Report skips in the parity step

A platform-gated case that stopped running on the row it exists for still
reports green with -q alone.

* Slice the error preference too

It is what turns the Write-Host throw into a dead script rather than a
skipped line, so restating it would be assuming the result.

* Say what the comments actually mean

* Make the console-less cases fail on a lost banner, not just a mangled one

* Stub the output sink in every harness that splices these scripts

The Write-Host rewrite left four spliced-source harnesses reaching
Write-StudioLine without defining it. An undefined command is a terminating
error, so each one either aborted or was swallowed by the harness's own catch,
and the test kept passing while no longer testing anything.

- test_windows_python_venv_hardening.py, partial-rollback case: the five-line
  split-move warning was lost. The assertion that "both halves are named" only
  stayed green because $existing is a prefix of the rollback dir, so it matched
  the dir= line instead. Pin it to the warning text.
- test_path_probe_access_denied.ps1, ownership guard: the catch scored the
  command-not-found as the intended failure and never reached Exit-SetupFailure.
  Pin the check to the EXIT-SETUP message.
- test_windows_installer_concurrency_guard.py: the decision block prints before
  Exit-InstallFailure, so on Windows the active case aborted at exit 1 and never
  produced RESULT:blocked.
- Studio.Setup.Vs2026.Tests.ps1: on a host without cmake,
  Ensure-BuildToolsForLlamaSourceBuild hits the sink first and the no-op case
  fails on the throw.

Also stub the three remaining harnesses that splice sink-calling helpers but do
not reach the sink on the paths they exercise today, so the next case added to
them cannot reintroduce this.

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-08-08 06:47:03 -07:00

330 lines
13 KiB
PowerShell

<#
Pester v5 unit tests for Write-StudioLine / step / substep /
Write-StudioStdoutMirror in studio/setup.ps1, guarding the desktop setup log
printing every step twice with the first copy split across two lines:
gpu
none (chat-only / GGUF)
gpu none (chat-only / GGUF)
Two causes. step/substep called Write-Host AND the console mirror, and the
CLI spawns setup.ps1 as `-Command "& '...' *>&1"`, so both reached the pipe.
And step's non-VT branch built one line from two Write-Host calls with
-NoNewline, which a redirected consumer splits at the record boundary.
Invariant now: exactly ONE sink. Redirected -> console handle. Interactive
-> Write-Host. Every other line in both entry scripts goes through
Write-StudioLine for the same reason: Write-Host is written by 5.1's console
host on the OEM code page, not by the UTF-8 writer bound to [Console]::Out,
so the banner and the footer used to arrive as U+FFFD.
Pure string formatting, so it runs on any pwsh host. Functions are extracted
and dot-sourced because setup.ps1 is a top-level installer; a missing one
FAILS loudly rather than silently passing.
#>
BeforeAll {
. (Join-Path $PSScriptRoot 'Get-FunctionSource.ps1')
$candidates = @(
$env:SETUP_PS1_PATH,
(Join-Path $PSScriptRoot '..\..\studio\setup.ps1')
) | Where-Object { $_ }
$script:SetupPs1 = $candidates | Where-Object { Test-Path -LiteralPath $_ } | Select-Object -First 1
if (-not $script:SetupPs1) { throw "Could not locate studio/setup.ps1 (set SETUP_PS1_PATH)." }
Write-Host "setup.ps1 under test: $script:SetupPs1"
$script:InstallPs1 = Join-Path $PSScriptRoot '..\..\install.ps1'
foreach ($fn in @('Get-StudioAnsi', 'Write-StudioLine', 'Write-StudioStdoutMirror', 'step', 'substep')) {
$src = Get-FunctionSource -Path $script:SetupPs1 -Name $fn
if (-not $src) { throw "Function '$fn' not found in $script:SetupPs1 - cannot test the real code." }
. ([scriptblock]::Create($src))
}
# Capture the console-handle sink without a real pipe, so the redirected
# path can be asserted from an ordinary interactive test host.
function Invoke-CapturingConsoleOut {
param(
[Parameter(Mandatory = $true)][scriptblock]$Body,
[Parameter(Mandatory = $true)][bool]$Redirected
)
$script:StudioStdoutRedirected = $Redirected
$previous = [Console]::Out
$writer = New-Object System.IO.StringWriter
try {
[Console]::SetOut($writer)
# 6>&1 folds Write-Host into the pipeline so a stray one on the
# redirected path is caught, not swallowed by the test host.
$hostRecords = & $Body 6>&1
} finally {
[Console]::SetOut($previous)
}
[pscustomobject]@{
Console = $writer.ToString()
HostRecordCount = @($hostRecords).Count
# Rendered to strings: 6>&1 yields InformationRecord objects, and the
# tests care about the text the user would have read.
HostRecords = @(@($hostRecords) | ForEach-Object { "$_" })
}
}
# Split on the real line separator only: the split-label bug produced a
# genuine newline, not a CR redraw.
#
# The leading comma is load-bearing. `return @($x)` unrolls a one-element
# array to a scalar, and a scalar string answers .Count = 1 while [0] gives
# its first CHARACTER, so a "one line, and it reads X" test would pass the
# count then compare against a single space.
function Get-EmittedLines {
param([string]$Text)
if ([string]::IsNullOrEmpty($Text)) { return , @() }
return , @($Text -split "`r?`n" | Where-Object { $_ -ne '' })
}
# Strip comments before asserting a construct is absent: the scripts under
# test describe -NoNewline in prose, which a naive match would hit.
function Get-CodeWithoutComments {
param([string]$Source)
$stripped = $Source -replace '(?m)#.*$', ''
return $stripped
}
}
Describe 'Write-StudioLine is the single sink for every non-step line' {
BeforeEach {
$script:StudioVtOk = $false
$env:NO_COLOR = $null
}
It 'writes to the console handle, and nothing to Write-Host, when redirected' {
$r = Invoke-CapturingConsoleOut -Redirected $true -Body {
Write-StudioLine "plain"
Write-StudioLine "colored" -ForegroundColor Red
}
(Get-EmittedLines $r.Console) | Should -Be @('plain', 'colored')
$r.HostRecordCount | Should -Be 0
}
It 'keeps the banner emoji and the U+2500 rule intact when redirected' {
$rule = [string]::new([char]0x2500, 52)
$sloth = [char]::ConvertFromUtf32(0x1F9A5)
$r = Invoke-CapturingConsoleOut -Redirected $true -Body {
Write-StudioLine (" " + $sloth + " Unsloth Studio Setup") -ForegroundColor Green
Write-StudioLine " $rule" -ForegroundColor DarkGray
}
$lines = Get-EmittedLines $r.Console
$lines.Count | Should -Be 2
$lines[0] | Should -Be " $sloth Unsloth Studio Setup"
$lines[1] | Should -Be " $rule"
}
It 'emits a blank line as a blank line, not as nothing' {
$r = Invoke-CapturingConsoleOut -Redirected $true -Body { Write-StudioLine "" }
$r.Console | Should -Not -BeNullOrEmpty
$r.Console.Trim() | Should -BeNullOrEmpty
}
It 'never leaks an ANSI escape onto the redirected sink' {
# Enable-StudioVirtualTerminal returns false without a console handle, so
# the colored branch must be unreachable there.
$r = Invoke-CapturingConsoleOut -Redirected $true -Body {
Write-StudioLine "warning" -ForegroundColor Yellow
}
$r.Console | Should -Not -Match ([regex]::Escape([char]27))
}
It 'stays on Write-Host, message intact, when attached to a console' {
$r = Invoke-CapturingConsoleOut -Redirected $false -Body {
Write-StudioLine "plain"
Write-StudioLine "colored" -ForegroundColor Red
}
$r.Console | Should -BeNullOrEmpty
$r.HostRecordCount | Should -Be 2
$r.HostRecords | Should -Be @('plain', 'colored')
}
It 'still prints an interactive blank line' {
$r = Invoke-CapturingConsoleOut -Redirected $false -Body { Write-StudioLine "" }
$r.HostRecordCount | Should -Be 1
}
It 'passes -ForegroundColor through only when the caller supplied one' {
# An omitted color must not become an empty string: Write-Host cannot
# bind that to ConsoleColor and the install would abort under "Stop".
{ Invoke-CapturingConsoleOut -Redirected $false -Body {
Write-StudioLine "no color here"
} } | Should -Not -Throw
}
}
Describe 'step / substep emit exactly one copy when stdout is redirected' {
BeforeEach {
$script:StudioVtOk = $false
$env:NO_COLOR = $null
}
It 'emits a step once, with label and value on the SAME line' {
$r = Invoke-CapturingConsoleOut -Redirected $true -Body {
step "gpu" "none (chat-only / GGUF)"
}
$lines = Get-EmittedLines $r.Console
$lines.Count | Should -Be 1
$lines[0] | Should -Be " gpu none (chat-only / GGUF)"
}
It 'emits a substep once' {
$r = Invoke-CapturingConsoleOut -Redirected $true -Body {
substep "installing OXC validator runtime..."
}
(Get-EmittedLines $r.Console).Count | Should -Be 1
}
It 'writes NOTHING through Write-Host when redirected (the second copy)' {
$r = Invoke-CapturingConsoleOut -Redirected $true -Body {
step "long paths" "enabled"
substep "detail"
}
$r.HostRecordCount | Should -Be 0
}
It 'keeps one line per step across a realistic run' {
$r = Invoke-CapturingConsoleOut -Redirected $true -Body {
step "gpu" "none (chat-only / GGUF)"
step "long paths" "enabled"
step "git" "git version 2.53.0.windows.2"
substep "setting up Python environment..."
}
(Get-EmittedLines $r.Console).Count | Should -Be 4
}
It 'truncates an over-long label to the 15-column field without wrapping' {
$r = Invoke-CapturingConsoleOut -Redirected $true -Body {
step "an-extremely-long-label" "value"
}
$lines = Get-EmittedLines $r.Console
$lines.Count | Should -Be 1
$lines[0] | Should -Be " an-extremely-lovalue"
}
}
Describe 'step / substep stay on Write-Host when attached to a console' {
BeforeEach {
$script:StudioVtOk = $false
$env:NO_COLOR = $null
}
It 'writes nothing to the console handle when NOT redirected' {
$r = Invoke-CapturingConsoleOut -Redirected $false -Body {
step "gpu" "none (chat-only / GGUF)"
}
$r.Console | Should -BeNullOrEmpty
}
It 'emits a step as a SINGLE Write-Host record (not -NoNewline label + value)' {
$r = Invoke-CapturingConsoleOut -Redirected $false -Body {
step "gpu" "none (chat-only / GGUF)"
}
$r.HostRecordCount | Should -Be 1
}
It 'emits a substep as a single Write-Host record' {
$r = Invoke-CapturingConsoleOut -Redirected $false -Body {
substep "detail"
}
$r.HostRecordCount | Should -Be 1
}
It 'still emits one record on the ANSI path' {
$script:StudioVtOk = $true
$r = Invoke-CapturingConsoleOut -Redirected $false -Body {
step "gpu" "ok"
substep "detail"
}
$r.HostRecordCount | Should -Be 2
}
It 'still emits one record with NO_COLOR set' {
$script:StudioVtOk = $true
$env:NO_COLOR = '1'
try {
$r = Invoke-CapturingConsoleOut -Redirected $false -Body {
step "gpu" "ok"
}
$r.HostRecordCount | Should -Be 1
} finally {
$env:NO_COLOR = $null
}
}
}
Describe 'Write-StudioStdoutMirror honors the resolved sink' {
It 'writes when the sink says redirected' {
$r = Invoke-CapturingConsoleOut -Redirected $true -Body {
Write-StudioStdoutMirror "hello"
}
(Get-EmittedLines $r.Console) | Should -Be @('hello')
}
It 'stays silent when the sink says interactive' {
$r = Invoke-CapturingConsoleOut -Redirected $false -Body {
Write-StudioStdoutMirror "hello"
}
$r.Console | Should -BeNullOrEmpty
}
}
Describe 'Source contracts that keep the fix from regressing' {
It 'resolves the redirected sink once, into a script-scoped variable' {
$source = Get-Content -Raw -LiteralPath $script:SetupPs1
$source | Should -Match '\$script:StudioStdoutRedirected\s*=\s*\[Console\]::IsOutputRedirected'
}
It 'no longer calls IsOutputRedirected from inside the mirror' {
# Reading it per-call let the sink disagree with the branch
# step/substep took; it must be resolved once, up front.
$src = Get-FunctionSource -Path $script:SetupPs1 -Name 'Write-StudioStdoutMirror'
$src | Should -Not -Match 'IsOutputRedirected'
}
It 'does not use -NoNewline in setup.ps1 step (record boundaries become newlines)' {
$src = Get-FunctionSource -Path $script:SetupPs1 -Name 'step'
(Get-CodeWithoutComments $src) | Should -Not -Match '-NoNewline'
}
It 'does not use -NoNewline in install.ps1 step either' {
$src = Get-FunctionSource -Path $script:InstallPs1 -Name 'step'
$src | Should -Not -BeNullOrEmpty
(Get-CodeWithoutComments $src) | Should -Not -Match '-NoNewline'
}
It 'defines Write-StudioLine in install.ps1 too, with the same body' {
# install.ps1 cannot dot-source setup.ps1, so it holds a copy. A copy
# that drifts is a copy that stops routing the installer's own banner.
$setup = ((Get-FunctionSource -Path $script:SetupPs1 -Name 'Write-StudioLine') -replace '\s+', ' ').Trim()
$install = ((Get-FunctionSource -Path $script:InstallPs1 -Name 'Write-StudioLine') -replace '\s+', ' ').Trim()
$setup | Should -Not -BeNullOrEmpty
$install | Should -Be $setup
}
It 'defines Write-StudioLine before the first line either script prints' {
# PowerShell resolves functions at call time, but a top-level call above
# the definition still fails. Comments are stripped first: both scripts
# name the helper in the prose above it.
foreach ($path in @($script:SetupPs1, $script:InstallPs1)) {
$source = Get-CodeWithoutComments (Get-Content -Raw -LiteralPath $path)
$definition = $source.IndexOf('function Write-StudioLine')
$definition | Should -BeGreaterThan -1
$source.IndexOf('Write-StudioLine') | Should -Be ($definition + 'function '.Length)
}
}
It 'sets the UTF-8 console encoding in both entry scripts' {
foreach ($path in @($script:SetupPs1, $script:InstallPs1)) {
$source = Get-Content -Raw -LiteralPath $path
$source | Should -Match '\[Console\]::OutputEncoding\s*=\s*\$_UnslothUtf8NoBom'
$source | Should -Match "PYTHONUTF8"
$source | Should -Match "PYTHONIOENCODING"
}
}
}