CodeNomad/scripts/build-webkit-read.ts
Pascal André 296af865a0
feat(tauri): embed native WebKit previews on macOS and Linux (#1301)
## Scope

Native browser previews, agent browser automation parity, and preview
capabilities across **Electron, Tauri Windows, Tauri macOS and Tauri
Linux**.

- Shared native embedding, geometry, overlay masking, session-owned
history and profile-scoped preview storage, isolated from the trusted
application renderer.
- Public `codenomad.browser`: open, navigate, snapshot, click, type,
select, press, scroll, hover, drag, wait and screenshot through the
authenticated backend and fenced native-parent channel.
- Six native viewport presets, WebKit HTTP/JavaScript mobile identity,
reset/reload and per-registration isolation. Native WebKit density/input
differences remain explicit.
- OAuth pop-ups with opener/postMessage and shared preview storage;
downloads with sanitized unique names; native dialogs and
owner/opener/renderer lifetime cleanup.
- Camera, microphone, location and notifications remain denied. Other
classified permissions retain native consent/selection and fail-closed
attribution. Guests receive no application privileges, app IPC, page
Node access or general evaluation bridge.
- Current-state developer documentation and user help in ten languages.

**Tauri macOS requires macOS 14 or newer; Electron keeps its own
platform baseline.**

## Host matrix

| Host | Native preview/history | Agent automation | Mobile profiles |
Pop-ups/downloads/dialogs |
| --- | --- | --- | --- | --- |
| Electron | Chromium | Supported | Chromium | Supported |
| Tauri Windows | WebView2 | Supported | Chromium | Supported |
| Tauri macOS 14+ | WKWebView | Supported | Native WebKit | Supported |
| Tauri Linux | WebKitGTK | Supported | Native WebKit | Supported |

## Qualification

- **Ubuntu 24.04 KDE VM:** real application built from the merged
sources, real backend and parent channel, private OpenCode daemon and
deterministic provider producing actual agent tool calls. Full browser
action sequence, OAuth opener messaging, downloaded-file contents,
native dialog acceptance and PNG passed under X11/Xwayland. Additional
native Wayland qualification uses an owned private KWin compositor;
native dialog operator coverage is recorded separately.
- **macOS 15.8 Intel VM:** same real-application/agent-tool path, all
actions and capability smoke passed with the built UI, private
profile/configuration and private OpenCode daemon. The app-window
nested-scroll check also qualifies the Cocoa wheel-coordinate
correction.
- **Windows:** Tauri release executable under its own
profile/configuration; all actions, OAuth, completed download, native
dialog and PNG passed through actual agent tool calls. The user's main
CodeNomad and shared OpenCode service remain untouched.
- Native CI qualifies Linux, macOS Intel, macOS ARM64 and Windows.
Authorized draft qualification uses no signing secrets.
- A **real Mac trial remains a release qualification step**. Graphical
macOS ARM64 and physical hardware capabilities retain their own
qualification scope; the capability matrix describes supported contracts
rather than exhaustive peripheral testing.

## Documented limits

- macOS first standalone hover can omit `pointerenter`; ⌘A selects
through the native responder while consuming the main A keydown.
- WebKit captures script-created closed shadow roots; declarative closed
shadow roots remain opaque.
- WebKitGTK preserves CVE-2025-13947 dropped-file protection; use the
page's native file input. No dropped URI text is converted into host
file access.
- GTK async clipboard reads lack requesting-frame attribution and are
refused; native Ctrl+V and clipboard writing remain available.
- WebKit keeps native screen/platform/pointer/hover/touch metadata.
WKWebView uses DPR 2.75 when its runtime-checked native selector exists;
GTK DPR follows monitor scale × fitted zoom. CSS dimensions remain
stable across app zoom/DPI. No JavaScript getter shims.
- Cross-origin frame contents retain the documented opaque automation
boundary. Page input timeouts have unknown outcomes and are never
replayed automatically.

## Integration

- Merge `ef69b4085`: `feat/tauri-webkit-parity`.
- Merge `e0d56d428`: `feat/tauri-webkit-capabilities` (`d58022b13`).
- Activation `3c68ab566`; Cocoa wheel correction `ecf5751ec`.
- The fetched `origin/dev` merges cleanly with this head. Pushed history
is preserved; no force-push or merge into `dev`.

The shared embedded-view layer remains available to remote-tabs work
while each consumer retains its own origin/profile/login/pairing/events
policy.
2026-11-08 23:46:06 +01:00

33 lines
4 KiB
TypeScript

import { buildSync } from "esbuild"
import { readFileSync, writeFileSync } from "node:fs"
import { gzipSync } from "node:zlib"
import { fileURLToPath } from "node:url"
import { ELEMENT_FUNCTIONS } from "../packages/server/src/automation/page/functions"
import { CLOSED_SHADOW_HOOK } from "../packages/server/src/automation/page/webkit/hook"
import { ACTION_AUDITOR, ACTION_PROGRAMS } from "../packages/tauri-app/tests/browser-parent/action-programs"
import ts from "typescript"
const root=new URL("../",import.meta.url)
const file=(path:string)=>new URL(path,root)
const ids=Object.keys(ELEMENT_FUNCTIONS) as Array<keyof typeof ELEMENT_FUNCTIONS>
const calls=`// Generated by scripts/build-webkit-read.ts.\n// @ts-nocheck\nexport const readCalls = {\n${ids.map(id=>`${JSON.stringify(id)}: function (...args) { const element=this.nodeType===1?this:this.parentElement; if (!element || !element.isConnected) throw new Error("codenomad:gone"); const document=element.ownerDocument; const { HTMLInputElement, HTMLTextAreaElement, HTMLLabelElement, HTMLSelectElement, HTMLElement, SVGElement, Node, Event, MouseEvent } = document.defaultView; const getSelection=()=>document.defaultView.getSelection(); ${ELEMENT_FUNCTIONS[id]} }`).join(",\n")}\n};\n`
writeFileSync(file("packages/server/src/automation/page/webkit/calls.generated.ts"),calls)
const bundled=buildSync({entryPoints:[fileURLToPath(file("packages/server/src/automation/page/webkit/read.ts"))],bundle:true,platform:"browser",format:"iife",globalName:"__codenomadReadModule",target:"es2021",minify:true,write:false})
const license=readFileSync(file("node_modules/dom-accessibility-api/LICENSE.md"),"utf8").replace(/\r\n/g,"\n")
const source=`/*! dom-accessibility-api 0.7.1\n${license}*/\n${bundled.outputFiles[0].text}\nreturn JSON.stringify(__codenomadReadModule.read(JSON.parse(request)));\n`
writeFileSync(file("packages/server/src/automation/page/webkit/read.generated.js"),source)
writeFileSync(file("packages/server/src/automation/page/webkit/LICENSE.dom-accessibility-api"),license)
const cString=(source:string)=>JSON.stringify(source).replace(/\?/g,"\\?")
writeFileSync(file("packages/tauri-app/src-tauri/src/embedded_view/page_read.generated.h"),`#pragma once\n/* Generated fixed isolated-world program; MIT notice is embedded in the source. */\nstatic const char CN_PAGE_READ[] = ${cString(source)};\nstatic const char CN_PAGE_HOOK[] = ${cString(CLOSED_SHADOW_HOOK)};\n`)
const extract=(path:string,name:string)=>{
const text=readFileSync(file(path),"utf8"), ast=ts.createSourceFile(path,text,ts.ScriptTarget.Latest,true)
for(const statement of ast.statements) if(ts.isVariableStatement(statement)) for(const declaration of statement.declarationList.declarations) if(ts.isIdentifier(declaration.name)&&declaration.name.text===name&&declaration.initializer&&ts.isNoSubstitutionTemplateLiteral(declaration.initializer)) return declaration.initializer.text
throw new Error(`Missing fixture ${name}`)
}
const preview=readFileSync(file("packages/ui/tests/browser/automation-preview.test.ts"),"utf8")
const ast=ts.createSourceFile("preview.ts",preview,ts.ScriptTarget.Latest,true)
let previewHtml=""
const visit=(node:ts.Node)=>{if(ts.isNoSubstitutionTemplateLiteral(node)&&node.text.includes("<!doctype html><main><form>"))previewHtml=node.text;ts.forEachChild(node,visit)};visit(ast)
const fixtures={preview:previewHtml,forms:extract("packages/ui/tests/browser/automation-forms.test.ts","FORM"),edges:extract("packages/ui/tests/browser/automation-forms.test.ts","EDGES"),inputs:extract("packages/ui/tests/browser/automation-input.test.ts","INPUTS")}
writeFileSync(file("packages/tauri-app/tests/browser-parent/read-fixtures.generated.json"),JSON.stringify(fixtures,null,2)+"\n")
writeFileSync(file("packages/tauri-app/tests/browser-parent/action-programs.generated.json"),JSON.stringify({auditor:ACTION_AUDITOR,programs:ACTION_PROGRAMS},null,2)+"\n")
console.log(JSON.stringify({library:"dom-accessibility-api@0.7.1",license:"MIT",bytes:Buffer.byteLength(source),gzip:gzipSync(source).length,fixtures:Object.keys(fixtures)}))