mirror of
https://github.com/anomalyco/opencode.git
synced 2026-09-09 19:25:36 +00:00
fix(app): stop transcript and inbox prefetch from inactive tabs (#46762)
This commit is contained in:
parent
fa4f8a66c2
commit
dfe3052bb6
6 changed files with 262 additions and 18 deletions
|
|
@ -85,14 +85,16 @@ Each tab scenario reports one sample, including its raw observations. Use Playwr
|
|||
|
||||
The tab-switch workload uses two equally long sessions: 200 user/assistant exchanges (400 messages) per tab. Every answer includes headings, emphasis, links, a blockquote, task and nested lists, an eight-row table, and four highlighted code fences (TSX, JSON, SQL, Bash), alongside the stress fixture's reasoning and tools. The mock API deliberately returns all 400 messages in one response so every scenario measures a long loaded history, not a short paginated tail. The viewport is fixed at 1440 x 900. Results include the fixture version, Markdown and serialized-message byte counts, and message-request count. These numbers are not directly comparable to the earlier 12-exchange source / 72-exchange destination fixture.
|
||||
|
||||
Cold means the destination transcript has never rendered in that fresh browser context. Warm means its complex answer was rendered and ready before switching away and back. Both use the app's normal restored-tab data prefetch, which completes before measurement; neither includes app startup, the source session's Markdown engine initialization, or a cold backend fetch. The suite asserts no message fetch during either measured switch. Setup waits for mounted Markdown to finish and for the review-pane width transition to complete. Service workers are blocked to exclude the web build's background asset precache from this renderer benchmark. Screenshots are attached after measurement for the first repetition; Playwright video and trace recording are disabled for this workload, while opt-in Chrome profiling remains available. For a baseline distribution, use `--repeat-each=20 --retries=0`, keep profiling disabled, and report the median and p95 of `firstCorrectObservedMs` separately from the three-observation `stableObservedMs`.
|
||||
Cold means the destination transcript has never loaded or rendered in that fresh browser context. Its measured switch includes one fixture message fetch. Warm means its complex answer was rendered and ready before switching away and back, and asserts no message fetch during the measured switch. Neither includes app startup or the source session's Markdown engine initialization. These cold results are not comparable to older prefetched cold-render results. Setup waits for mounted Markdown to finish and for the review-pane width transition to complete. Service workers are blocked to exclude the web build's background asset precache from this renderer benchmark. Screenshots are attached after measurement for the first repetition; Playwright video and trace recording are disabled for this workload, while opt-in Chrome profiling remains available. For a baseline distribution, use `--repeat-each=20 --retries=0`, keep profiling disabled, and report the median and p95 of `firstCorrectObservedMs` separately from the three-observation `stableObservedMs`.
|
||||
|
||||
```sh
|
||||
bunx playwright test --config e2e/performance/playwright.config.ts \
|
||||
timeline/session-tab-switch-benchmark.spec.ts --repeat-each=20 --retries=0
|
||||
```
|
||||
|
||||
**The tab-switch fixture is not an end-to-end cold-data benchmark.** It prefetches destination messages and returns full history. Measure cold API navigation, Home-row opening, and prefetched-but-unvisited tabs separately with normal pagination. Do not combine these entry paths or compare different transports and machine-load periods as one experiment.
|
||||
**The tab-switch fixture returns full history, not normal pagination.** Measure cold API navigation, Home-row opening, and restored-but-unvisited tabs separately with normal pagination. Do not combine these entry paths or compare different transports and machine-load periods as one experiment.
|
||||
|
||||
`inactive-tab-prefetch-benchmark.spec.ts` restores eight tabs with normal 20-message pages (44 parts and 139,257 response bytes per page). It gates heavy responses independently until every tab's attention callback has run, then measures selection with ready answer Markdown and bottom anchoring. A separate case closes an inactive tab before releasing the responses. The fixture reports speculative transcript/inbox reads, request concurrency, response bytes, and activation latency. Set `OPENCODE_PERFORMANCE_MEMORY=1` only in separate retention runs; those force GC before selection and must not be mixed into clean timing results. The scope is the production browser renderer, not total desktop memory. Live background events and eviction of previously visited transcripts are separate workloads.
|
||||
|
||||
Keep one-off reports, recorded results, and traces outside git, in the ignored `e2e/performance/results/` directory or an external artifact directory. Preserve raw observations locally and publish anonymized summaries and charts in the PR description, not as committed experiment files.
|
||||
|
||||
|
|
@ -104,7 +106,7 @@ bun run bench:tabs
|
|||
|
||||
This runs only the tab-switch benchmark against the production build with 20 serial repetitions and no retries. It prints the median (mean of the two middle values for even sample counts) and nearest-rank p95 for `firstCorrectObservedMs` and `stableObservedMs` per scenario. Only records whose benchmark and Playwright statuses are passed and whose two metrics are finite enter the summary. Test and record statuses, missing records, and excluded samples are reported separately.
|
||||
|
||||
For fresh entry paths, run `bun run bench:entry` from `packages/app`. It uses the same production, serial-repetition, and reporting defaults. The cases open an empty draft from the actual Home button, create a draft with the titlebar plus from an active session, and open a cold paginated session from Home. Draft readiness requires a focused editable composer, the expected model, project control, and new tab; typing and absence of backend mutations are checked afterward. Session readiness requires the latest group, ready answer Markdown, and bottom anchoring. These cases are separate from prefetched tab remounts.
|
||||
For fresh entry paths, run `bun run bench:entry` from `packages/app`. It uses the same production, serial-repetition, and reporting defaults. The cases open an empty draft from the actual Home button, create a draft with the titlebar plus from an active session, and open a cold paginated session from Home. Draft readiness requires a focused editable composer, the expected model, project control, and new tab; typing and absence of backend mutations are checked afterward. Session readiness requires the latest group, ready answer Markdown, and bottom anchoring. These cases are separate from cached tab remounts.
|
||||
|
||||
For milestone charts, rerun frozen builds with one workload and counterbalanced serial order. Do not connect historical medians from different transports, preparation, or machine-load periods. Show samples or ranges, name the checkpoints accurately, and distinguish experimental build snapshots from Git commits.
|
||||
|
||||
|
|
|
|||
|
|
@ -0,0 +1,166 @@
|
|||
import { benchmark, expect } from "../benchmark"
|
||||
import { mockOpenCodeServer } from "../../utils/mock-server"
|
||||
import { expectSessionTitle } from "../../utils/waits"
|
||||
import { fixture } from "./session-timeline-stress.fixture"
|
||||
import { messages } from "./session-tab-switch.fixture"
|
||||
import { installStressSessionTabs, stressSessionHref } from "./timeline-test-helpers"
|
||||
import { measureSessionSwitch, waitForStableTimeline } from "./session-tab-switch-probe"
|
||||
|
||||
const sessions = Array.from({ length: 8 }, (_, index) => ({
|
||||
...fixture.sessions[0],
|
||||
id: `ses_prefetch_${index}`,
|
||||
title: `Renderer review ${index}`,
|
||||
}))
|
||||
// A normal first page, not the full-history response used by the tab-switch benchmark.
|
||||
const pages = Object.fromEntries(
|
||||
sessions.map((session) => [
|
||||
session.id,
|
||||
messages[fixture.targetID].slice(-20).map((message) => ({ ...message, id: `${message.id}_${session.id}` })),
|
||||
]),
|
||||
)
|
||||
const workload = {
|
||||
sessions: sessions.length,
|
||||
messagesPerPage: 20,
|
||||
payloadBytes: Object.fromEntries(
|
||||
sessions.map((session) => [
|
||||
session.id,
|
||||
Buffer.byteLength(JSON.stringify({ data: pages[session.id].toReversed(), cursor: {} })),
|
||||
]),
|
||||
),
|
||||
partsPerPage: pages[sessions[0].id].reduce(
|
||||
(count, message) => count + (message.type === "assistant" ? message.content.length : 1),
|
||||
0,
|
||||
),
|
||||
events: 0,
|
||||
}
|
||||
type ProbeWindow = Window & { __prefetchBodies?: Record<string, number> }
|
||||
|
||||
benchmark.use({ viewport: { width: 1440, height: 900 }, video: "off", trace: "off", serviceWorkers: "block" })
|
||||
|
||||
for (const close of [false, true]) {
|
||||
benchmark(
|
||||
`inactive tab prefetch: ${close ? "close before response" : "activate after restore"}`,
|
||||
async ({ page, report }, testInfo) => {
|
||||
const gate = Promise.withResolvers<void>()
|
||||
const reads: string[] = []
|
||||
const inboxReads: string[] = []
|
||||
const pending = new Set<string>()
|
||||
const concurrency = { peak: 0 }
|
||||
const mutations: string[] = []
|
||||
const errors: string[] = []
|
||||
page.on("pageerror", (error) => errors.push(error.message))
|
||||
page.on("response", (response) => {
|
||||
if (new URL(response.url()).pathname.startsWith("/api/") && !response.ok())
|
||||
errors.push(`HTTP ${response.status()}: ${response.url()}`)
|
||||
})
|
||||
page.on("request", (request) => {
|
||||
const path = new URL(request.url()).pathname
|
||||
if (request.method() === "DELETE" || /\/(interrupt|prompt)$/.test(path)) mutations.push(request.url())
|
||||
const inbox = path.match(/^\/api\/session\/([^/]+)\/inbox$/)
|
||||
if (request.method() === "GET" && inbox) inboxReads.push(inbox[1])
|
||||
})
|
||||
await page.addInitScript(() => {
|
||||
const host = window as ProbeWindow
|
||||
host.__prefetchBodies = {}
|
||||
const text = Response.prototype.text
|
||||
Response.prototype.text = async function () {
|
||||
const body = await text.call(this)
|
||||
if (this.url) {
|
||||
const path = new URL(this.url).pathname
|
||||
host.__prefetchBodies![path] = (host.__prefetchBodies![path] ?? 0) + 1
|
||||
}
|
||||
return body
|
||||
}
|
||||
})
|
||||
await mockOpenCodeServer(page, {
|
||||
...fixture,
|
||||
sessions,
|
||||
pageMessages: (id) => ({ items: pages[id] ?? [] }),
|
||||
beforeMessagesResponse: ({ sessionID }) => (sessionID === sessions[0].id ? Promise.resolve() : gate.promise),
|
||||
onMessages: ({ sessionID, phase }) => {
|
||||
if (phase === "end") return void pending.delete(sessionID)
|
||||
reads.push(sessionID)
|
||||
pending.add(sessionID)
|
||||
concurrency.peak = Math.max(concurrency.peak, pending.size)
|
||||
},
|
||||
})
|
||||
await installStressSessionTabs(page, { sessionIDs: sessions.map((session) => session.id) })
|
||||
const cdp = await page.context().newCDPSession(page)
|
||||
await cdp.send("Performance.enable")
|
||||
await page.goto(stressSessionHref(sessions[0].id))
|
||||
await expectSessionTitle(page, sessions[0].title)
|
||||
await waitForStableTimeline(page, pages[sessions[0].id].at(-2)!.id)
|
||||
// Every inactive tab's scheduled attention request must finish. This gates on the
|
||||
// same production callback as prefetch, without a sleep or waiting for a removed read.
|
||||
await page.waitForFunction(
|
||||
(ids) => ids.every((id) => (window as ProbeWindow).__prefetchBodies![`/api/session/${id}/form`] > 0),
|
||||
sessions.slice(1).map((session) => session.id),
|
||||
)
|
||||
const speculativeReads = reads.filter((id) => id !== sessions[0].id)
|
||||
const speculativeInboxReads = inboxReads.filter((id) => id !== sessions[0].id).length
|
||||
const closed = sessions.at(-1)!
|
||||
if (close) {
|
||||
const tab = page
|
||||
.locator("[data-titlebar-tab-slot]")
|
||||
.filter({ has: page.locator(`a[href="${stressSessionHref(closed.id)}"]`) })
|
||||
await tab.getByRole("button", { name: "Close tab", exact: true }).click()
|
||||
await expect(tab).toHaveCount(0)
|
||||
}
|
||||
gate.resolve()
|
||||
await page.waitForFunction(
|
||||
(ids) => ids.every((id) => (window as ProbeWindow).__prefetchBodies![`/api/session/${id}/message`] > 0),
|
||||
reads,
|
||||
)
|
||||
await expectSessionTitle(page, sessions[0].title)
|
||||
const heap =
|
||||
process.env.OPENCODE_PERFORMANCE_MEMORY === "1"
|
||||
? await cdp.send("HeapProfiler.collectGarbage").then(() => cdp.send("Runtime.getHeapUsage"))
|
||||
: undefined
|
||||
const task =
|
||||
(await cdp.send("Performance.getMetrics")).metrics.find((metric) => metric.name === "TaskDuration")!.value *
|
||||
1000
|
||||
const before = reads.length
|
||||
const target = sessions[1]
|
||||
const result = await measureSessionSwitch(page, {
|
||||
destinationIDs: pages[target.id].map((message) => message.id),
|
||||
sourceIDs: pages[sessions[0].id].map((message) => message.id),
|
||||
lastID: pages[target.id].at(-2)!.id,
|
||||
requiredPartID: `${pages[target.id].at(-1)!.id}:text:0`,
|
||||
href: stressSessionHref(target.id),
|
||||
switch: async () => {
|
||||
await page.locator(`[data-slot="titlebar-tabs"] a[href="${stressSessionHref(target.id)}"]`).click()
|
||||
await expectSessionTitle(page, target.title)
|
||||
},
|
||||
})
|
||||
await expect(
|
||||
page.locator(`[data-timeline-part-id="${pages[target.id].at(-1)!.id}:text:0"] [data-component="markdown"]`),
|
||||
).toHaveAttribute("data-markdown-ready", "")
|
||||
expect(result.firstCorrectObservedMs).not.toBeNull()
|
||||
expect(mutations).toEqual([])
|
||||
expect(errors).toEqual([])
|
||||
report(
|
||||
{
|
||||
speculativeReads: speculativeReads.length,
|
||||
speculativeInboxReads,
|
||||
speculativePayloadBytes: speculativeReads.reduce((bytes, id) => bytes + workload.payloadBytes[id], 0),
|
||||
peakMessageRequests: concurrency.peak,
|
||||
closedSessionReads: close ? reads.filter((id) => id === closed.id).length : undefined,
|
||||
activationReads: reads.length - before,
|
||||
startupTaskMs: task,
|
||||
retainedHeap: heap,
|
||||
...result,
|
||||
},
|
||||
{
|
||||
...workload,
|
||||
close,
|
||||
gc: heap ? "forced retention; timing diagnostic only" : "natural; clean timing",
|
||||
transport: "playwright-route",
|
||||
browser: page.context().browser()!.version(),
|
||||
scope: "production app renderer; not total desktop RAM",
|
||||
},
|
||||
)
|
||||
if (testInfo.repeatEachIndex === 0) await page.screenshot({ path: testInfo.outputPath("destination.png") })
|
||||
await cdp.detach()
|
||||
},
|
||||
)
|
||||
}
|
||||
|
|
@ -48,20 +48,27 @@ benchmark("samples cached session repaint after the click", async ({ page, repor
|
|||
expect(result.samples.length).toBeGreaterThan(0)
|
||||
})
|
||||
|
||||
benchmark("prefetches every open session tab", async ({ page, report }) => {
|
||||
const prefetched = new Set<string>()
|
||||
benchmark("loads only the selected restored tab's transcript", async ({ page, report }) => {
|
||||
const loaded = new Set<string>()
|
||||
await mockStressTimeline(page, {
|
||||
onMessages: (input) => {
|
||||
if (!input.before && input.phase === "start") prefetched.add(input.sessionID)
|
||||
if (!input.before && input.phase === "start") loaded.add(input.sessionID)
|
||||
},
|
||||
})
|
||||
await installStressSessionTabs(page, {
|
||||
sessionIDs: [fixture.sourceID, fixture.targetID, fixture.childID],
|
||||
})
|
||||
await installTimelineSettings(page)
|
||||
const attention = Promise.all(
|
||||
[fixture.targetID, fixture.childID].map((id) =>
|
||||
page.waitForResponse((response) => new URL(response.url()).pathname === `/api/session/${id}/form`),
|
||||
),
|
||||
)
|
||||
await page.goto(stressSessionHref(fixture.sourceID))
|
||||
await expectSessionTitle(page, fixture.expected.sourceTitle)
|
||||
|
||||
await expect.poll(() => prefetched.has(fixture.childID)).toBe(true)
|
||||
report({ prefetched: [...prefetched] })
|
||||
await attention
|
||||
await waitForStableTimeline(page, fixture.expected.sourceMessageIDs.at(-1)!)
|
||||
expect([...loaded]).toEqual([fixture.sourceID])
|
||||
report({ loaded: [...loaded] })
|
||||
})
|
||||
|
|
|
|||
|
|
@ -49,7 +49,7 @@ scenarios.forEach((scenario) => {
|
|||
|
||||
expect(result.firstCorrectObservedMs).not.toBeNull()
|
||||
expect(result.stableObservedMs).not.toBeNull()
|
||||
expect(requests).toHaveLength(requestsBefore)
|
||||
expect(requests).toHaveLength(requestsBefore + (scenario.cache === "cold" ? 1 : 0))
|
||||
await expectReadyTimeline(page, fixture.targetID)
|
||||
report(
|
||||
{
|
||||
|
|
@ -65,7 +65,7 @@ scenarios.forEach((scenario) => {
|
|||
browserVersion: page.context().browser()!.version(),
|
||||
serviceWorkers: "blocked",
|
||||
reviewFiles: scenario.review === "closed" ? 0 : reviewDiffs.length,
|
||||
data: "prefetched",
|
||||
data: scenario.cache === "cold" ? "on-demand" : "cached",
|
||||
transport: process.env.OPENCODE_PERFORMANCE_HTTP_FIXTURE === "1" ? "http" : "playwright-route",
|
||||
inputEvent: "mousedown",
|
||||
requireReadyAnswer: true,
|
||||
|
|
@ -97,16 +97,11 @@ async function prepareSessionTabs(page: Page) {
|
|||
})
|
||||
await installTimelineSettings(page)
|
||||
await installStressSessionTabs(page)
|
||||
// Restored tabs prefetch their data even when their transcript has never rendered.
|
||||
const prefetch = page.waitForResponse((response) =>
|
||||
new URL(response.url()).pathname.endsWith(`/session/${fixture.targetID}/message`),
|
||||
)
|
||||
await page.goto(stressSessionHref(fixture.sourceID))
|
||||
expect(await (await prefetch).finished()).toBeNull()
|
||||
await expectSessionTitle(page, fixture.expected.sourceTitle)
|
||||
await expectReadyTimeline(page, fixture.sourceID)
|
||||
await expect(page.locator(`[data-timeline-part-id="${expected[fixture.targetID].answerID}"]`)).toHaveCount(0)
|
||||
expect(requests.toSorted()).toEqual([fixture.sourceID, fixture.targetID].toSorted())
|
||||
expect(requests).toEqual([fixture.sourceID])
|
||||
return requests
|
||||
}
|
||||
|
||||
|
|
|
|||
75
packages/app/e2e/regression/inactive-tab-prefetch.spec.ts
Normal file
75
packages/app/e2e/regression/inactive-tab-prefetch.spec.ts
Normal file
|
|
@ -0,0 +1,75 @@
|
|||
import { expect, test } from "@playwright/test"
|
||||
import { mockOpenCodeServer } from "../utils/mock-server"
|
||||
import { expectSessionTitle } from "../utils/waits"
|
||||
import { fixture } from "../performance/timeline/session-timeline-stress.fixture"
|
||||
import { installStressSessionTabs, stressSessionHref } from "../performance/timeline/timeline-test-helpers"
|
||||
|
||||
test("inactive tabs load attention, but read transcript and inbox only on selection", async ({ page }) => {
|
||||
const reads: string[] = []
|
||||
const mutations: string[] = []
|
||||
const errors: string[] = []
|
||||
page.on("pageerror", (error) => errors.push(error.message))
|
||||
page.on("response", (response) => {
|
||||
if (new URL(response.url()).pathname.startsWith("/api/") && !response.ok())
|
||||
errors.push(`HTTP ${response.status()}: ${response.url()}`)
|
||||
})
|
||||
const state = { text: "Original fixture answer" }
|
||||
page.on("request", (request) => {
|
||||
const path = new URL(request.url()).pathname
|
||||
if (!path.startsWith("/api/")) return
|
||||
if (request.method() === "GET") reads.push(path)
|
||||
if (request.method() === "DELETE" || /\/(interrupt|prompt)$/.test(path)) mutations.push(path)
|
||||
})
|
||||
await mockOpenCodeServer(page, {
|
||||
...fixture,
|
||||
pageMessages: (id) => ({
|
||||
items: [
|
||||
{ id: `msg_${id}_user`, type: "user", text: "Review the renderer change", time: { created: 1 } },
|
||||
{
|
||||
id: `msg_${id}_assistant`,
|
||||
type: "assistant",
|
||||
agent: "build",
|
||||
model: { id: "claude-opus-4-6", providerID: "opencode" },
|
||||
content: [{ type: "text", text: state.text }],
|
||||
time: { created: 2, completed: 3 },
|
||||
},
|
||||
],
|
||||
}),
|
||||
})
|
||||
await installStressSessionTabs(page, { sessionIDs: [fixture.sourceID, fixture.targetID, fixture.childID] })
|
||||
const attention = Promise.all(
|
||||
[fixture.targetID, fixture.childID].flatMap((id) =>
|
||||
["permission", "form"].map((kind) =>
|
||||
page.waitForResponse((response) => new URL(response.url()).pathname === `/api/session/${id}/${kind}`),
|
||||
),
|
||||
),
|
||||
)
|
||||
await page.goto(stressSessionHref(fixture.sourceID))
|
||||
await expectSessionTitle(page, fixture.expected.sourceTitle)
|
||||
await expect(page.locator(`[data-timeline-part-id="msg_${fixture.sourceID}_assistant:text:0"]`)).toContainText(
|
||||
state.text,
|
||||
)
|
||||
await attention
|
||||
const child = page
|
||||
.locator("[data-titlebar-tab-slot]")
|
||||
.filter({ has: page.locator(`a[href="${stressSessionHref(fixture.childID)}"]`) })
|
||||
await child.getByRole("button", { name: "Close tab", exact: true }).click()
|
||||
await expect(child).toHaveCount(0)
|
||||
state.text = "Latest fixture answer after tab restoration"
|
||||
await page.locator(`[data-slot="titlebar-tabs"] a[href="${stressSessionHref(fixture.targetID)}"]`).click()
|
||||
await expectSessionTitle(page, fixture.expected.targetTitle)
|
||||
await expect(page.locator(`[data-timeline-part-id="msg_${fixture.targetID}_assistant:text:0"]`)).toContainText(
|
||||
state.text,
|
||||
)
|
||||
for (const id of [fixture.sourceID, fixture.targetID]) {
|
||||
expect(reads.filter((path) => path === `/api/session/${id}/message`)).toHaveLength(1)
|
||||
expect(reads.filter((path) => path === `/api/session/${id}/inbox`)).toHaveLength(1)
|
||||
}
|
||||
expect(
|
||||
reads.filter(
|
||||
(path) => path === `/api/session/${fixture.childID}/message` || path === `/api/session/${fixture.childID}/inbox`,
|
||||
),
|
||||
).toEqual([])
|
||||
expect(mutations).toEqual([])
|
||||
expect(errors).toEqual([])
|
||||
})
|
||||
|
|
@ -138,8 +138,7 @@ function SessionTabEntry(props: {
|
|||
() =>
|
||||
void Promise.allSettled([
|
||||
ctx.data.session.sync(value.id, { children: true }),
|
||||
ctx.data.session.pending.sync(value.id),
|
||||
ctx.data.session.message.sync(value.id),
|
||||
// The selected timeline loads transcript and inbox data; inactive tabs need only attention and metadata.
|
||||
ctx.data.session.permission.sync(value.id),
|
||||
ctx.data.session.form.sync(value.id),
|
||||
]),
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue