chore(config): add disableIncrementalCache to skip populating remote R2 incremental cache (#4623)

This commit is contained in:
Huang Xin 2026-06-17 09:51:17 +08:00 committed by GitHub
parent 6514d4aa58
commit 5e217544f2
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
5 changed files with 101 additions and 1 deletions

View file

@ -43,7 +43,10 @@
- [TTS sync paragraph+RSVP (#3235, PR #4576)](tts-sync-paragraph-rsvp-3235.md) — TTS-is-clock follow: canonical `tts-position{cfi,kind:word|sentence,sectionIndex,sequence}`; in-mode 🔊 audio toggle (`build{Paragraph,Rsvp}TtsSpeakDetail`, live-range gate); **current word/sentence highlight painted on the overlay CLONE via CSS Custom Highlight API** (no DOM mutation, spans inline; offsets relative to para-start map 1:1 to clone, `getTextSubRange` reuse, index-tagged vs stale); kind-gating `decideParagraphTtsHighlight` (Edge word wins, skip coarse sentence); `::highlight()` from `ttsHighlightOptions`
## Build & Vendoring
- [Turbopack build-cache OOM + gated Docker standalone (#4619)](turbopack-build-cache-oom-docker-standalone.md) — interrupted-build partial `turbopackFileSystemCacheForBuild` cache → 42 workers/18GB-swap freeze (clean build=~6.5GB); disabled the flag; `output:'standalone'` gated on `BUILD_STANDALONE` (Docker-only); tauri CI uses `next dev` (config-independent)
- [Deps/security override workflow](deps-security-overrides-workflow.md) — fix transitive npm Dependabot alerts: main monorepo overrides live in `pnpm-workspace.yaml` (NOT root package.json); `packages/tauri-plugins` is a SEPARATE submodule project w/ own lockfile + `minimumReleaseAge` (main workspace has no age gate); bound 0.x overrides like `vite`; verify via test+lint+build-web. PR #4618 (esbuild 0.28.1, vitest 4.1.9)
- [R2 rclone CreateBucket 403 (#4588)](r2-rclone-createbucket-403.md) — single-file `rclone copyto`/`moveto` probes CreateBucket → 403 on object-scoped R2 token; use a directory `rclone copy` (or `no_check_bucket=true`); broke nightly assemble, not the release flow
- [Deploy workers.dev SNI-block + proxy](deploy-workers-dev-sni-proxy.md) — pnpm deploy crash (CN): workers.dev SNI-blocked (DoH useless), R2 populate WS hangs even via proxy; shipped fix = `dangerous.disableIncrementalCache:true` in open-next.config (stock deploy skips populate; readest has no ISR so runtime no-op)
- [pdfjs vendor wasm decoders](pdfjs-vendor-wasm-decoders.md) — scanned PDFs blank in CI build only (0.11.2 regression); pdfjs 5.7.x moved JBIG2 to `jbig2.wasm`, `copy-pdfjs-wasm` allow-list dropped it; `cpx` no-errors on empty glob; local stale `public/vendor` (gitignored, not refreshed by `tauri build`) masked it; fix = copy `wasm/*`
## Platform Compat

View file

@ -0,0 +1,24 @@
---
name: deploy-workers-dev-sni-proxy
description: "pnpm deploy crashes in China — workers.dev SNI-blocked, wrangler ws WebSocket bypasses http_proxy; fix = NODE_OPTIONS preload"
metadata:
node_type: memory
type: project
originSessionId: 65342d98-7939-41ed-9e10-2efc466946b1
---
`pnpm deploy` (and `pnpm upload`) crashed for chrox (behind GFW, Privoxy at `http://127.0.0.1:8118`) with an **unhandled `ws` `'error'` event → Node process crash** (ETIMEDOUT to Facebook/Vultr/Twitter IPs).
**Trigger:** `opennextjs-cloudflare deploy`/`upload` ALWAYS runs `populateCache({target:"remote"})` BEFORE the real deploy (no skip flag; only `cacheChunkSize`/`env` knobs). That step calls wrangler's `unstable_startWorker({remote:true})`, which opens a **WebSocket** to a `*.workers.dev` edge host. (`preview` uses `target:"local"` → unaffected.)
**Root cause (two layers):**
1. `*.workers.dev` is **SNI-blocked** by the GFW, not merely DNS-poisoned. Proof: encrypted DoH gives the REAL Cloudflare IPs (104.18.x), but a *direct* TLS connect to that correct IP with SNI=workers.dev is still `Connection reset by peer` before TLS starts. So **DoH/dnscrypt-proxy does NOT help** — the connection must avoid being made directly at all.
2. wrangler's REST calls honor `http_proxy` (undici `ProxyAgent`/`EnvHttpProxyAgent`), but the raw `ws` handshake falls back to `https.globalAgent` and **ignores proxy env**. So it connects directly → SNI reset → crash. The crash fires async (unhandled WS 'error'), so `opennextjs-cloudflare deploy`'s `await` can't catch it.
**Attempt 1 — proxy preload (tried, then REMOVED):** a zero-dep preload that replaced `https.globalAgent` with a `CONNECT`-tunnel agent (CONNECT hides the SNI = defeats the block + does remote DNS; loopback bypassed so the local populate worker on 127.0.0.1 still works). Verified `https.get('https://workers.dev')` → 301 via proxy. This got PAST the WebSocket crash — the local populate worker started and enumerated all 17 cache assets — **BUT the actual R2 writes through the remote binding then timed out** ("Failed to send request to R2 worker: aborted due to timeout", retrying forever). The proxy establishes the connection but can't reliably carry the sustained cache-write traffic. So the preload alone is NOT sufficient. Deleted it.
**Attempt 2 — replicate `wrangler deploy` in the npm script (tried, then reverted):** skip populate by bypassing `opennextjs-cloudflare deploy` and running `CLOUDFLARE_LOAD_DEV_VARS_FROM_DOT_ENV=false OPEN_NEXT_DEPLOY=true wrangler deploy` directly (traced from `runWrangler`: stock deploy's real step is plain `wrangler deploy` vs `wrangler.toml` which has `main=.open-next/worker.js`+all bindings; no generated config/skew mapping; the env flag stops wrangler 4.x auto-loading `.env`/`.dev.vars` into the worker — OpenNext's adapter handles env). Works, but hacky (replicates internals, drift risk).
**Fix that SHIPPED — config flag (cleanest).** populateCache is gated by `if (!config.dangerous?.disableIncrementalCache && incrementalCache)`. So in `open-next.config.ts`: `config.dangerous = { ...config.dangerous, disableIncrementalCache: true }`. This makes the STOCK `opennextjs-cloudflare deploy`/`upload` skip populate (no script hack, no env flag, no drift) — reverted package.json to stock. Caveat: it's the SAME flag the runtime reads, so it ALSO disables the runtime incremental cache — **but readest uses ZERO ISR (no `revalidate`/`unstable_cache`/`'use cache'`/`generateStaticParams`), so runtime caching is a no-op anyway → no real loss.** Re-enable = delete the one line (from a network that can reach the CF edge). `defineCloudflareConfig` returns `OpenNextConfig` (broad type; `dangerous.disableIncrementalCache?: boolean` exists), tsgo+biome clean. `preview` was always fine (local populate).
Related: [[turbopack-build-cache-oom-docker-standalone]], [[r2-rclone-createbucket-403]].

View file

@ -0,0 +1,45 @@
---
name: deps-security-overrides-workflow
description: "How to fix transitive npm Dependabot alerts in the readest monorepo (pnpm-workspace overrides, where config lives, tauri-plugins is separate)"
metadata:
node_type: memory
type: reference
originSessionId: c61e7dd2-4033-4bd1-8f32-22056e4ef322
---
Fixing transitive npm Dependabot security alerts (manifest `pnpm-lock.yaml`).
**Where pnpm config lives (non-obvious):** the MAIN monorepo's `overrides`,
`patchedDependencies`, `onlyBuiltDependencies`, `allowBuilds` are in
**`pnpm-workspace.yaml`** (newer pnpm style) — NOT the root `package.json`
(root `package.json` has no `pnpm` section). The root `pnpm-lock.yaml` is what
Dependabot scans; alerts report manifest `pnpm-lock.yaml` = this root lockfile.
**`packages/tauri-plugins` is a SEPARATE project**, not part of the main pnpm
workspace. It's a git submodule (`tauri-plugins-workspace`) with its OWN
`pnpm-lock.yaml` and its own `package.json` `pnpm.overrides` +
`minimumReleaseAge: 4320`. The `minimumReleaseAge` (3-day age gate) applies ONLY
there — the main monorepo has NO age gate, so `^X` specs resolve to the very
latest matching version. Dependabot does not scan the tauri-plugins lockfile.
`pnpm-workspace.yaml` `packages:` = `apps/*`, send-email worker, extensions,
`packages/foliate-js` (NOT tauri-plugins).
**Recipe for a transitive advisory:**
1. Add `pkg: '>=X.Y.Z'` to the `overrides:` block in `pnpm-workspace.yaml`
(forces all transitive instances up). For risky 0.x packages, BOUND it like
the existing `vite: '>=7.3.2 <8'` (e.g. `esbuild: '>=0.28.1 <0.29'`).
2. For packages that are also DIRECT deps, bump the spec in
`apps/readest-app/package.json` too (e.g. the vitest family:
`vitest`, `@vitest/browser-playwright`, `@vitest/browser-webdriverio`,
`@vitest/coverage-v8` — move in lockstep).
3. `pnpm install`, then `grep -oE "pkg@[0-9.]+" pnpm-lock.yaml | sort -u` to
confirm no vulnerable versions remain.
4. Verify: `pnpm test` + `pnpm lint` + `pnpm build-web` (the last exercises
esbuild in the OpenNext/Cloudflare bundle path).
**Override applicability:** an override forces a transitive version regardless
of the parent's declared range ONLY when the package is a regular dep (no peer
warning). esbuild is a regular dep of vite; vite 7.3.x pins esbuild `^0.27.0`
but esbuild 0.28.x is API-compatible for vite's usage (0.28 changelog = install
integrity + minifier/codegen fixes). Verified via PR #4618 (alerts #238/#239
esbuild→0.28.1, #240 @vitest/browser→4.1.9).

View file

@ -0,0 +1,20 @@
---
name: turbopack-build-cache-oom-docker-standalone
description: "Turbopack build-cache OOM that froze the Mac, and the gated Docker standalone image (PR"
metadata:
node_type: memory
type: project
originSessionId: 41c10f07-ba55-451b-89b4-7e13a5fa3087
---
PR #4619 (merged, squash 6514d4aa5): web Docker production image → Next.js standalone, plus disabling the experimental Turbopack build cache.
**RAM-meltdown root cause (cost a force-reboot to find).** `experimental.turbopackFileSystemCacheForBuild: true` (beta) is the culprit, NOT the standalone config and NOT the compiler. When a `pnpm build-web` (Turbopack production build) is interrupted mid-compile (e.g. SIGTERM), it leaves a **partial** build cache in `.next/`. The next build on top of that poisoned cache fans out to ~42 worker processes and drives swap to ~18 GB (macOS auto-grows the swapfile) on a 16 GB M1 Pro → thrash → freeze. A *clean* Turbopack `next build` of readest peaks at only ~6.5 GB and fits fine. **Proven by A/B from a cold `rm -rf .next`:** both `output: undefined` and `output: 'standalone'` complete in ~48 s at ~6.5 GB with flat swap; only the warm/partial-cache run exploded.
- Fixes: removed `turbopackFileSystemCacheForBuild` (kept `turbopackFileSystemCacheForDev`); removed the `pull-request.yml` step that cached `.next/cache` for it (now dead).
- Rule: **after interrupting a local build, `rm -rf .next` before retrying.** Never run an unbounded `pnpm build-web` on the dev machine.
**Watchdog gotcha.** A host-side memory watchdog that polls `ps`/recursive-`pgrep` over the build tree STARVES under thrash (its own syscalls block on paging) and can't fire the kill. Working pattern: sample every 1 s, decision FIRST using only cheap `sysctl vm.swapusage` + `vm_stat`, kill EARLY (avail < 3.8 GB or swap_used > 2.5 GB), `nice -n 19` the build. Even so, prefer Docker (capped VM RAM) or CI over local full builds.
**Docker standalone, gated.** `output: 'standalone'` + `outputFileTracingRoot` (monorepo root) are gated on a `BUILD_STANDALONE` env flag set ONLY in the `Dockerfile` build stage (`ENV BUILD_STANDALONE=true` before `pnpm build-web`). Every other path keeps original output: Tauri `export`, local `build-web`/dev (`undefined`), and the Cloudflare/OpenNext deploy (`undefined` in config, but OpenNext forces standalone via `setStandaloneBuildMode``NEXT_PRIVATE_STANDALONE`, so idempotent). `production-stage` copies `.next/standalone` (→ `apps/readest-app/server.js` + hoisted `node_modules`) + `.next/static` + `public`, runs non-root `node`, entrypoint `node apps/readest-app/server.js`. `docker-image.yml` builds it on merge only (not PRs), so verify the gate locally.
**Tauri CI red herring.** `build_tauri_app`'s `run tauri tests` step (`scripts/test-tauri.sh`) uses `next dev` + `tauri dev --features webdriver` — both config-output-independent, so next.config `output`/tracing changes can't affect it. Normally 23 min (Swatinem/rust-cache restores crates); a cold cache makes it 1520 min of "Compiling …" that looks stuck but isn't. See [[r2-rclone-createbucket-403]] for other build/deploy CI notes.

View file

@ -1,6 +1,14 @@
import { defineCloudflareConfig } from '@opennextjs/cloudflare';
import r2IncrementalCache from '@opennextjs/cloudflare/overrides/incremental-cache/r2-incremental-cache';
export default defineCloudflareConfig({
const config = defineCloudflareConfig({
incrementalCache: r2IncrementalCache,
});
// Skip the deploy-time R2 incremental-cache populate step. The app uses no ISR /
// `revalidate` / `unstable_cache`, so the runtime incremental cache is a no-op
// anyway. Remove this line to re-enable (deploy from a network that can reach the
// Cloudflare edge).
config.dangerous = { ...config.dangerous, disableIncrementalCache: true };
export default config;