fix(docker): unbreak large uploads and self-hosted cloud sync (#6100)

* fix(web): stop truncating request bodies over 10MB (#6091)

`middleware.ts` matches /api/*, so Next buffers a clone of every non-GET
request body. Past `experimental.proxyClientMaxBodySize` (10MB by default)
`getCloneableBody` pushes null into both the middleware's stream and the one
`finalize()` splices back over the IncomingMessage, so the route handler sees a
short body ending cleanly: no error, no 413, just a stored file of exactly
10,485,760 bytes. `/api/send/inbox/file` accepts 40MB, so an inbox EPUB over
10MB was already being stored corrupt.

The clone is buffered in memory before any handler can reject it, which makes
the ceiling an unauthenticated memory budget on every /api/* route. Only the
self-hosted image, which may proxy whole book files through the Node server,
gets real headroom; `standaloneOutput` is the gate because BUILD_STANDALONE is
set by the Dockerfile alone and web.readest.com runs on Cloudflare/Vercel.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(settings): stop gating cloud sync as Premium when self-hosted (#6093)

Two causes of the same symptom on a self-hosted Docker deployment.

The badge keyed off sign-in, not entitlement:

  !user || (userProfilePlan !== undefined && !isCloudSyncPremium)

`!user` short-circuits, so a signed-out self-hoster — the normal state right
after `docker compose up` — got a Premium chip on WebDAV, Google Drive, S3 and
OneDrive. The rows opened fine and sync was never paused, but the chip reads as
"not available in this deployment". `shouldShowCloudProviderBadge` decides on
entitlement instead: self-hosting unlocks with or without a signed-in user.

`SELF_HOSTED` also only reached the container through `compose.yaml`
(`${SELF_HOSTED:-true}`), which landed in #5996. A compose file copied before
that survives every `docker compose pull`, so those deployments came up gated
with nothing on screen to explain why. The image now defaults it on: it is the
self-hosted artifact, web.readest.com never runs it, and an operator selling
plans still overrides with SELF_HOSTED=false.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* docs(settings): clarify that the tier chip still gates signed-out visitors

The doc comment described what changed rather than what the predicate does, so
"entitlement is the deciding input, not sign-in state" read as if signed-out
users stopped being gated. They are not: a signed-out visitor to a hosted
deployment is not entitled, so the row is badged and stays behind
canToggleCloudProvider and the upgrade route, exactly as before. Only
entitlement — which self-hosting confers — removes the chip.

Assert the pairing in one test so a future refactor cannot decouple the badge
from the gate.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Huang Xin 2026-09-06 23:56:25 +08:00 • committed by GitHub
parent 2af468e3bf
commit 731b13ab6f
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
8 changed files with 181 additions and 9 deletions

View file

@ -59,6 +59,14 @@ FROM docker.io/library/node:24-slim@sha256:24dc26ef1e3c3690f27ebc4136c9c186c3133
ENV NODE_ENV=production
ENV PORT=3000
ENV HOSTNAME=0.0.0.0
# This image is the self-hosted artifact — web.readest.com runs on
# Cloudflare/Vercel, never on it — so premium features are unlocked by default
# rather than left to the operator's compose file. `compose.yaml` passes
# `SELF_HOSTED: ${SELF_HOSTED:-true}`, but that line only landed in #5996: a
# compose file copied before it survives every `docker compose pull`, so the
# container came up gated with nothing on screen to explain why (#6093).
# An operator running a hosted service that sells plans sets SELF_HOSTED=false.
ENV SELF_HOSTED=true
WORKDIR /app
# Monorepo-rooted standalone tree: server.js + hoisted, traced node_modules.
COPY --from=build --chown=node:node /app/apps/readest-app/.next/standalone ./

View file

@ -48,6 +48,26 @@ const nextConfig = {
// a build interrupted mid-compile leaves a partial cache that the next
// build mishandles, fanning out workers until it exhausts RAM.
turbopackFileSystemCacheForDev: true,
// `middleware.ts` matches /api/*, so Next buffers a clone of every request
// body to let both the middleware and the route handler read it. Past this
// limit the clone is truncated and the handler sees a short body ending
// cleanly — an upload that "succeeds" while storing a corrupt file, with no
// error to the client (#6091).
//
// The clone is buffered in memory before any handler can reject it, which
// makes this ceiling an unauthenticated memory budget on every /api/* route
// — so only the self-hosted image, which may proxy whole book files through
// the Node server, gets real headroom. `standaloneOutput` is the right gate
// because BUILD_STANDALONE is set by the Dockerfile alone: web.readest.com
// runs on Cloudflare/Vercel, never on that image. (This value is baked into
// required-server-files.json at build time, so a runtime env var could not
// do the same job.)
//
// 32MB is deliberately under SEND_INBOX_FILE_MAX_BYTES (40MB): the only
// build that both runs the Node server and takes that route is a plain
// `next start`, since Cloudflare and Vercel never reach this code path, so
// the tighter budget is worth more there than the last 8MB of an upload.
proxyClientMaxBodySize: standaloneOutput ? 1024 * 1024 * 1024 : 32 * 1024 * 1024,
},
// Configure assetPrefix or else the server won't properly resolve your assets.
assetPrefix: '',

View file

@ -3,6 +3,7 @@ import {
canToggleCloudProvider,
getReadestCloudRowStatus,
getThirdPartyRowStatus,
shouldShowCloudProviderBadge,
} from '@/components/settings/integrations/cloudSyncStatus';
const _ = (key: string) => key;
@ -140,3 +141,42 @@ describe('canToggleCloudProvider', () => {
);
});
});
describe('shouldShowCloudProviderBadge', () => {
test('signed out on a hosted deployment sees the tier chip and stays gated', () => {
// The chip and the gate have to move together: a signed-out visitor to a
// hosted deployment is not entitled, so the row is badged AND unusable.
expect(
shouldShowCloudProviderBadge({ signedIn: false, planLoading: false, isPremium: false }),
).toBe(true);
expect(canToggleCloudProvider({ isPremium: false, isConfigured: true, isEnabled: false })).toBe(
false,
);
});
test('signed out on a self-hosted deployment does not (#6093)', () => {
// Self-hosting unlocks cloud sync with or without a signed-in user, so the
// chip would label a feature the operator already has as premium.
expect(
shouldShowCloudProviderBadge({ signedIn: false, planLoading: false, isPremium: true }),
).toBe(false);
});
test('stays hidden while a signed-in user plan is still resolving', () => {
expect(
shouldShowCloudProviderBadge({ signedIn: true, planLoading: true, isPremium: false }),
).toBe(false);
});
test('a resolved free plan sees the tier chip', () => {
expect(
shouldShowCloudProviderBadge({ signedIn: true, planLoading: false, isPremium: false }),
).toBe(true);
});
test('an entitled user never sees it', () => {
expect(
shouldShowCloudProviderBadge({ signedIn: true, planLoading: false, isPremium: true }),
).toBe(false);
});
});

View file

@ -0,0 +1,24 @@
import { readFileSync } from 'node:fs';
import path from 'node:path';
import { fileURLToPath } from 'node:url';
import { describe, expect, test } from 'vitest';
const repoRoot = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '../../../..');
const dockerfile = readFileSync(path.join(repoRoot, 'Dockerfile'), 'utf8');
const productionStage = dockerfile.slice(dockerfile.indexOf('AS production-stage'));
describe('Docker image runtime defaults', () => {
/**
* The published image is the self-hosted artifact — web.readest.com runs on
* Cloudflare/Vercel, never on this image — so it defaults `SELF_HOSTED` on
* itself instead of relying on the operator's compose file. `compose.yaml`
* passes `SELF_HOSTED: ${SELF_HOSTED:-true}`, but that line only landed in
* #5996: a compose file copied before it stays on disk through every
* `docker compose pull`, so the container came up gated with no way for the
* operator to know why (#6093). An operator selling plans still overrides it
* with `SELF_HOSTED=false`.
*/
test('unlocks premium features without an operator-supplied env var', () => {
expect(productionStage).toMatch(/^ENV SELF_HOSTED=true$/m);
});
});

View file

@ -1,5 +1,17 @@
import { describe, expect, test } from 'vitest';
import { afterEach, describe, expect, test, vi } from 'vitest';
import nextConfig from '../../next.config.mjs';
import { SEND_INBOX_FILE_MAX_BYTES } from '@/services/constants';
/** Next's own `getCloneableBody` fallback (`DEFAULT_BODY_CLONE_SIZE_LIMIT`). */
const NEXT_DEFAULT_BODY_CLONE_LIMIT = 10 * 1024 * 1024;
/** Re-evaluate next.config.mjs as the Docker image builds it. */
const loadSelfHostedConfig = async () => {
vi.stubEnv('NEXT_PUBLIC_APP_PLATFORM', 'web');
vi.stubEnv('BUILD_STANDALONE', 'true');
vi.resetModules();
return (await import('../../next.config.mjs')).default;
};
describe('Next.js static asset headers', () => {
test('keeps bundled workers cross-origin isolated', async () => {
@ -15,3 +27,40 @@ describe('Next.js static asset headers', () => {
});
});
});
describe('Proxy request body clone limit', () => {
// `middleware.ts` matches every /api/* route, and Next buffers a clone of the
// request body so both the middleware and the route handler can read it. Past
// `experimental.proxyClientMaxBodySize` the clone is truncated and the route
// handler sees a short body with a clean `end` — a silently corrupt upload,
// not an error (#6091).
afterEach(() => {
vi.unstubAllEnvs();
vi.resetModules();
});
test('every build raises it past the truncating default', () => {
expect(nextConfig.experimental?.proxyClientMaxBodySize).toBeGreaterThan(
NEXT_DEFAULT_BODY_CLONE_LIMIT,
);
});
// The clone is buffered in memory before any handler can reject it, so the
// ceiling is also an unauthenticated memory budget on every /api/* route.
// Only the self-hosted image — which may proxy whole book files through the
// Node server — gets real headroom, so it is also the only build guaranteed
// to carry the largest body our own routes accept. Cloudflare and Vercel
// never reach this code path, leaving `next start` as the one place where the
// narrower budget can still clip an inbox upload.
test('only the self-hosted Docker build covers the largest API route body', async () => {
const selfHosted = await loadSelfHostedConfig();
expect(selfHosted.output).toBe('standalone');
expect(selfHosted.experimental?.proxyClientMaxBodySize).toBeGreaterThanOrEqual(
SEND_INBOX_FILE_MAX_BYTES,
);
expect(selfHosted.experimental?.proxyClientMaxBodySize).toBeGreaterThan(
nextConfig.experimental?.proxyClientMaxBodySize as number,
);
});
});

View file

@ -58,6 +58,7 @@ import {
canToggleCloudProvider,
getReadestCloudRowStatus,
getThirdPartyRowStatus,
shouldShowCloudProviderBadge,
} from './integrations/cloudSyncStatus';
import {
getCloudSyncProviders,
@ -142,13 +143,13 @@ const IntegrationsPanel: React.FC = () => {
// back on. The `?? 'free'` keeps the (re-gated) loading state non-premium.
const { userProfilePlan, customizationPurchased } = useQuotaStats();
const isCloudSyncPremium = isCloudSyncAllowed(userProfilePlan ?? 'free', customizationPurchased);
// Only surface the tier chip to users who cannot use the feature yet — signed
// out (known immediately), or signed in on a plan without cloud sync (known
// once the plan resolves). An entitled user already has it, so the badge is
// noise. Suppressing it while a signed-in user's plan is still loading avoids
// flashing the chip at a premium user on every open.
const premiumBadge =
!user || (userProfilePlan !== undefined && !isCloudSyncPremium) ? _('Premium') : undefined;
const premiumBadge = shouldShowCloudProviderBadge({
signedIn: !!user,
planLoading: userProfilePlan === undefined,
isPremium: isCloudSyncPremium,
})
? _('Premium')
: undefined;
const [subPage, setSubPage] = useState<SubPage>(null);

View file

@ -62,6 +62,29 @@ export interface CanToggleCloudProviderInputs {
export const canToggleCloudProvider = (s: CanToggleCloudProviderInputs): boolean =>
(s.isPremium && s.isConfigured) || s.isEnabled;
export interface CloudProviderBadgeInputs {
signedIn: boolean;
/** Plan still resolving from the JWT (signed-in only). */
planLoading: boolean;
/** Cloud sync is usable right now — a plan, the unlock, or a self-hosted deployment. */
isPremium: boolean;
}
/**
* Whether a third-party provider row carries the tier chip. Only for users who
* cannot use the feature: an entitled user already has it, so the badge is
* noise. Suppressed while a signed-in user's plan is still resolving, which
* would otherwise flash the chip at a premium user on every open.
*
* Signed out on a hosted deployment still carries the chip, and the row stays
* gated behind {@link canToggleCloudProvider} and the upgrade route — that
* cohort is not entitled. What #6093 changed is which input removes the chip:
* entitlement, not sign-in. `!user` used to short-circuit it ON, so a
* self-hoster who could already open every provider was told it was Premium.
*/
export const shouldShowCloudProviderBadge = (s: CloudProviderBadgeInputs): boolean =>
!s.isPremium && (!s.signedIn || !s.planLoading);
export const getThirdPartyRowStatus = (_: TranslationFunc, s: ThirdPartyRowInputs): string => {
if (!s.enabled) return s.configured ? _('Configured') : _('Not connected');
if (s.paused) return _('Paused — plan required');

View file

@ -50,9 +50,16 @@ docker compose up -d
```
this pulls `${READEST_IMAGE}` (default: `ghcr.io/readest/readest:latest`) instead of building the client locally.
the web client now reads `SUPABASE_PUBLIC_URL`, `SUPABASE_ANON_KEY`, `API_BASE_URL`, `OBJECT_STORAGE_TYPE`, `STORAGE_FIXED_QUOTA`, and `TRANSLATION_FIXED_QUOTA` from runtime
the web client now reads `SUPABASE_PUBLIC_URL`, `SUPABASE_ANON_KEY`, `API_BASE_URL`, `OBJECT_STORAGE_TYPE`, `SELF_HOSTED`, `STORAGE_FIXED_QUOTA`, and `TRANSLATION_FIXED_QUOTA` from runtime
container env, so custom self-hosted values work with pulled images.
`SELF_HOSTED` unlocks every premium feature — third-party cloud sync (WebDAV,
Google Drive, S3, OneDrive), offline Read Aloud downloads, Send to Readest —
with or without a signed-in user. the image defaults it to `true`, so a pulled
image is unlocked even if your `compose.yaml` predates the variable. set
`SELF_HOSTED=false` in `docker/.env` only if you run a hosted service that sells
the plans.
if you prefer Docker Hub, set `READEST_IMAGE` in `docker/.env`, for example:
```env