mirror of
https://github.com/readest/readest.git
synced 2026-10-19 10:13:29 +00:00
fix(docker): unbreak large uploads and self-hosted cloud sync (#6100)
* fix(web): stop truncating request bodies over 10MB (#6091) `middleware.ts` matches /api/*, so Next buffers a clone of every non-GET request body. Past `experimental.proxyClientMaxBodySize` (10MB by default) `getCloneableBody` pushes null into both the middleware's stream and the one `finalize()` splices back over the IncomingMessage, so the route handler sees a short body ending cleanly: no error, no 413, just a stored file of exactly 10,485,760 bytes. `/api/send/inbox/file` accepts 40MB, so an inbox EPUB over 10MB was already being stored corrupt. The clone is buffered in memory before any handler can reject it, which makes the ceiling an unauthenticated memory budget on every /api/* route. Only the self-hosted image, which may proxy whole book files through the Node server, gets real headroom; `standaloneOutput` is the gate because BUILD_STANDALONE is set by the Dockerfile alone and web.readest.com runs on Cloudflare/Vercel. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(settings): stop gating cloud sync as Premium when self-hosted (#6093) Two causes of the same symptom on a self-hosted Docker deployment. The badge keyed off sign-in, not entitlement: !user || (userProfilePlan !== undefined && !isCloudSyncPremium) `!user` short-circuits, so a signed-out self-hoster — the normal state right after `docker compose up` — got a Premium chip on WebDAV, Google Drive, S3 and OneDrive. The rows opened fine and sync was never paused, but the chip reads as "not available in this deployment". `shouldShowCloudProviderBadge` decides on entitlement instead: self-hosting unlocks with or without a signed-in user. `SELF_HOSTED` also only reached the container through `compose.yaml` (`${SELF_HOSTED:-true}`), which landed in #5996. A compose file copied before that survives every `docker compose pull`, so those deployments came up gated with nothing on screen to explain why. The image now defaults it on: it is the self-hosted artifact, web.readest.com never runs it, and an operator selling plans still overrides with SELF_HOSTED=false. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs(settings): clarify that the tier chip still gates signed-out visitors The doc comment described what changed rather than what the predicate does, so "entitlement is the deciding input, not sign-in state" read as if signed-out users stopped being gated. They are not: a signed-out visitor to a hosted deployment is not entitled, so the row is badged and stays behind canToggleCloudProvider and the upgrade route, exactly as before. Only entitlement — which self-hosting confers — removes the chip. Assert the pairing in one test so a future refactor cannot decouple the badge from the gate. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
parent
2af468e3bf
commit
731b13ab6f
8 changed files with 181 additions and 9 deletions
|
|
@ -59,6 +59,14 @@ FROM docker.io/library/node:24-slim@sha256:24dc26ef1e3c3690f27ebc4136c9c186c3133
|
|||
ENV NODE_ENV=production
|
||||
ENV PORT=3000
|
||||
ENV HOSTNAME=0.0.0.0
|
||||
# This image is the self-hosted artifact — web.readest.com runs on
|
||||
# Cloudflare/Vercel, never on it — so premium features are unlocked by default
|
||||
# rather than left to the operator's compose file. `compose.yaml` passes
|
||||
# `SELF_HOSTED: ${SELF_HOSTED:-true}`, but that line only landed in #5996: a
|
||||
# compose file copied before it survives every `docker compose pull`, so the
|
||||
# container came up gated with nothing on screen to explain why (#6093).
|
||||
# An operator running a hosted service that sells plans sets SELF_HOSTED=false.
|
||||
ENV SELF_HOSTED=true
|
||||
WORKDIR /app
|
||||
# Monorepo-rooted standalone tree: server.js + hoisted, traced node_modules.
|
||||
COPY --from=build --chown=node:node /app/apps/readest-app/.next/standalone ./
|
||||
|
|
|
|||
|
|
@ -48,6 +48,26 @@ const nextConfig = {
|
|||
// a build interrupted mid-compile leaves a partial cache that the next
|
||||
// build mishandles, fanning out workers until it exhausts RAM.
|
||||
turbopackFileSystemCacheForDev: true,
|
||||
// `middleware.ts` matches /api/*, so Next buffers a clone of every request
|
||||
// body to let both the middleware and the route handler read it. Past this
|
||||
// limit the clone is truncated and the handler sees a short body ending
|
||||
// cleanly — an upload that "succeeds" while storing a corrupt file, with no
|
||||
// error to the client (#6091).
|
||||
//
|
||||
// The clone is buffered in memory before any handler can reject it, which
|
||||
// makes this ceiling an unauthenticated memory budget on every /api/* route
|
||||
// — so only the self-hosted image, which may proxy whole book files through
|
||||
// the Node server, gets real headroom. `standaloneOutput` is the right gate
|
||||
// because BUILD_STANDALONE is set by the Dockerfile alone: web.readest.com
|
||||
// runs on Cloudflare/Vercel, never on that image. (This value is baked into
|
||||
// required-server-files.json at build time, so a runtime env var could not
|
||||
// do the same job.)
|
||||
//
|
||||
// 32MB is deliberately under SEND_INBOX_FILE_MAX_BYTES (40MB): the only
|
||||
// build that both runs the Node server and takes that route is a plain
|
||||
// `next start`, since Cloudflare and Vercel never reach this code path, so
|
||||
// the tighter budget is worth more there than the last 8MB of an upload.
|
||||
proxyClientMaxBodySize: standaloneOutput ? 1024 * 1024 * 1024 : 32 * 1024 * 1024,
|
||||
},
|
||||
// Configure assetPrefix or else the server won't properly resolve your assets.
|
||||
assetPrefix: '',
|
||||
|
|
|
|||
|
|
@ -3,6 +3,7 @@ import {
|
|||
canToggleCloudProvider,
|
||||
getReadestCloudRowStatus,
|
||||
getThirdPartyRowStatus,
|
||||
shouldShowCloudProviderBadge,
|
||||
} from '@/components/settings/integrations/cloudSyncStatus';
|
||||
|
||||
const _ = (key: string) => key;
|
||||
|
|
@ -140,3 +141,42 @@ describe('canToggleCloudProvider', () => {
|
|||
);
|
||||
});
|
||||
});
|
||||
|
||||
describe('shouldShowCloudProviderBadge', () => {
|
||||
test('signed out on a hosted deployment sees the tier chip and stays gated', () => {
|
||||
// The chip and the gate have to move together: a signed-out visitor to a
|
||||
// hosted deployment is not entitled, so the row is badged AND unusable.
|
||||
expect(
|
||||
shouldShowCloudProviderBadge({ signedIn: false, planLoading: false, isPremium: false }),
|
||||
).toBe(true);
|
||||
expect(canToggleCloudProvider({ isPremium: false, isConfigured: true, isEnabled: false })).toBe(
|
||||
false,
|
||||
);
|
||||
});
|
||||
|
||||
test('signed out on a self-hosted deployment does not (#6093)', () => {
|
||||
// Self-hosting unlocks cloud sync with or without a signed-in user, so the
|
||||
// chip would label a feature the operator already has as premium.
|
||||
expect(
|
||||
shouldShowCloudProviderBadge({ signedIn: false, planLoading: false, isPremium: true }),
|
||||
).toBe(false);
|
||||
});
|
||||
|
||||
test('stays hidden while a signed-in user plan is still resolving', () => {
|
||||
expect(
|
||||
shouldShowCloudProviderBadge({ signedIn: true, planLoading: true, isPremium: false }),
|
||||
).toBe(false);
|
||||
});
|
||||
|
||||
test('a resolved free plan sees the tier chip', () => {
|
||||
expect(
|
||||
shouldShowCloudProviderBadge({ signedIn: true, planLoading: false, isPremium: false }),
|
||||
).toBe(true);
|
||||
});
|
||||
|
||||
test('an entitled user never sees it', () => {
|
||||
expect(
|
||||
shouldShowCloudProviderBadge({ signedIn: true, planLoading: false, isPremium: true }),
|
||||
).toBe(false);
|
||||
});
|
||||
});
|
||||
|
|
|
|||
24
apps/readest-app/src/__tests__/docker-image-defaults.test.ts
Normal file
24
apps/readest-app/src/__tests__/docker-image-defaults.test.ts
Normal file
|
|
@ -0,0 +1,24 @@
|
|||
import { readFileSync } from 'node:fs';
|
||||
import path from 'node:path';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
import { describe, expect, test } from 'vitest';
|
||||
|
||||
const repoRoot = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '../../../..');
|
||||
const dockerfile = readFileSync(path.join(repoRoot, 'Dockerfile'), 'utf8');
|
||||
const productionStage = dockerfile.slice(dockerfile.indexOf('AS production-stage'));
|
||||
|
||||
describe('Docker image runtime defaults', () => {
|
||||
/**
|
||||
* The published image is the self-hosted artifact — web.readest.com runs on
|
||||
* Cloudflare/Vercel, never on this image — so it defaults `SELF_HOSTED` on
|
||||
* itself instead of relying on the operator's compose file. `compose.yaml`
|
||||
* passes `SELF_HOSTED: ${SELF_HOSTED:-true}`, but that line only landed in
|
||||
* #5996: a compose file copied before it stays on disk through every
|
||||
* `docker compose pull`, so the container came up gated with no way for the
|
||||
* operator to know why (#6093). An operator selling plans still overrides it
|
||||
* with `SELF_HOSTED=false`.
|
||||
*/
|
||||
test('unlocks premium features without an operator-supplied env var', () => {
|
||||
expect(productionStage).toMatch(/^ENV SELF_HOSTED=true$/m);
|
||||
});
|
||||
});
|
||||
|
|
@ -1,5 +1,17 @@
|
|||
import { describe, expect, test } from 'vitest';
|
||||
import { afterEach, describe, expect, test, vi } from 'vitest';
|
||||
import nextConfig from '../../next.config.mjs';
|
||||
import { SEND_INBOX_FILE_MAX_BYTES } from '@/services/constants';
|
||||
|
||||
/** Next's own `getCloneableBody` fallback (`DEFAULT_BODY_CLONE_SIZE_LIMIT`). */
|
||||
const NEXT_DEFAULT_BODY_CLONE_LIMIT = 10 * 1024 * 1024;
|
||||
|
||||
/** Re-evaluate next.config.mjs as the Docker image builds it. */
|
||||
const loadSelfHostedConfig = async () => {
|
||||
vi.stubEnv('NEXT_PUBLIC_APP_PLATFORM', 'web');
|
||||
vi.stubEnv('BUILD_STANDALONE', 'true');
|
||||
vi.resetModules();
|
||||
return (await import('../../next.config.mjs')).default;
|
||||
};
|
||||
|
||||
describe('Next.js static asset headers', () => {
|
||||
test('keeps bundled workers cross-origin isolated', async () => {
|
||||
|
|
@ -15,3 +27,40 @@ describe('Next.js static asset headers', () => {
|
|||
});
|
||||
});
|
||||
});
|
||||
|
||||
describe('Proxy request body clone limit', () => {
|
||||
// `middleware.ts` matches every /api/* route, and Next buffers a clone of the
|
||||
// request body so both the middleware and the route handler can read it. Past
|
||||
// `experimental.proxyClientMaxBodySize` the clone is truncated and the route
|
||||
// handler sees a short body with a clean `end` — a silently corrupt upload,
|
||||
// not an error (#6091).
|
||||
afterEach(() => {
|
||||
vi.unstubAllEnvs();
|
||||
vi.resetModules();
|
||||
});
|
||||
|
||||
test('every build raises it past the truncating default', () => {
|
||||
expect(nextConfig.experimental?.proxyClientMaxBodySize).toBeGreaterThan(
|
||||
NEXT_DEFAULT_BODY_CLONE_LIMIT,
|
||||
);
|
||||
});
|
||||
|
||||
// The clone is buffered in memory before any handler can reject it, so the
|
||||
// ceiling is also an unauthenticated memory budget on every /api/* route.
|
||||
// Only the self-hosted image — which may proxy whole book files through the
|
||||
// Node server — gets real headroom, so it is also the only build guaranteed
|
||||
// to carry the largest body our own routes accept. Cloudflare and Vercel
|
||||
// never reach this code path, leaving `next start` as the one place where the
|
||||
// narrower budget can still clip an inbox upload.
|
||||
test('only the self-hosted Docker build covers the largest API route body', async () => {
|
||||
const selfHosted = await loadSelfHostedConfig();
|
||||
|
||||
expect(selfHosted.output).toBe('standalone');
|
||||
expect(selfHosted.experimental?.proxyClientMaxBodySize).toBeGreaterThanOrEqual(
|
||||
SEND_INBOX_FILE_MAX_BYTES,
|
||||
);
|
||||
expect(selfHosted.experimental?.proxyClientMaxBodySize).toBeGreaterThan(
|
||||
nextConfig.experimental?.proxyClientMaxBodySize as number,
|
||||
);
|
||||
});
|
||||
});
|
||||
|
|
|
|||
|
|
@ -58,6 +58,7 @@ import {
|
|||
canToggleCloudProvider,
|
||||
getReadestCloudRowStatus,
|
||||
getThirdPartyRowStatus,
|
||||
shouldShowCloudProviderBadge,
|
||||
} from './integrations/cloudSyncStatus';
|
||||
import {
|
||||
getCloudSyncProviders,
|
||||
|
|
@ -142,13 +143,13 @@ const IntegrationsPanel: React.FC = () => {
|
|||
// back on. The `?? 'free'` keeps the (re-gated) loading state non-premium.
|
||||
const { userProfilePlan, customizationPurchased } = useQuotaStats();
|
||||
const isCloudSyncPremium = isCloudSyncAllowed(userProfilePlan ?? 'free', customizationPurchased);
|
||||
// Only surface the tier chip to users who cannot use the feature yet — signed
|
||||
// out (known immediately), or signed in on a plan without cloud sync (known
|
||||
// once the plan resolves). An entitled user already has it, so the badge is
|
||||
// noise. Suppressing it while a signed-in user's plan is still loading avoids
|
||||
// flashing the chip at a premium user on every open.
|
||||
const premiumBadge =
|
||||
!user || (userProfilePlan !== undefined && !isCloudSyncPremium) ? _('Premium') : undefined;
|
||||
const premiumBadge = shouldShowCloudProviderBadge({
|
||||
signedIn: !!user,
|
||||
planLoading: userProfilePlan === undefined,
|
||||
isPremium: isCloudSyncPremium,
|
||||
})
|
||||
? _('Premium')
|
||||
: undefined;
|
||||
|
||||
const [subPage, setSubPage] = useState<SubPage>(null);
|
||||
|
||||
|
|
|
|||
|
|
@ -62,6 +62,29 @@ export interface CanToggleCloudProviderInputs {
|
|||
export const canToggleCloudProvider = (s: CanToggleCloudProviderInputs): boolean =>
|
||||
(s.isPremium && s.isConfigured) || s.isEnabled;
|
||||
|
||||
export interface CloudProviderBadgeInputs {
|
||||
signedIn: boolean;
|
||||
/** Plan still resolving from the JWT (signed-in only). */
|
||||
planLoading: boolean;
|
||||
/** Cloud sync is usable right now — a plan, the unlock, or a self-hosted deployment. */
|
||||
isPremium: boolean;
|
||||
}
|
||||
|
||||
/**
|
||||
* Whether a third-party provider row carries the tier chip. Only for users who
|
||||
* cannot use the feature: an entitled user already has it, so the badge is
|
||||
* noise. Suppressed while a signed-in user's plan is still resolving, which
|
||||
* would otherwise flash the chip at a premium user on every open.
|
||||
*
|
||||
* Signed out on a hosted deployment still carries the chip, and the row stays
|
||||
* gated behind {@link canToggleCloudProvider} and the upgrade route — that
|
||||
* cohort is not entitled. What #6093 changed is which input removes the chip:
|
||||
* entitlement, not sign-in. `!user` used to short-circuit it ON, so a
|
||||
* self-hoster who could already open every provider was told it was Premium.
|
||||
*/
|
||||
export const shouldShowCloudProviderBadge = (s: CloudProviderBadgeInputs): boolean =>
|
||||
!s.isPremium && (!s.signedIn || !s.planLoading);
|
||||
|
||||
export const getThirdPartyRowStatus = (_: TranslationFunc, s: ThirdPartyRowInputs): string => {
|
||||
if (!s.enabled) return s.configured ? _('Configured') : _('Not connected');
|
||||
if (s.paused) return _('Paused — plan required');
|
||||
|
|
|
|||
|
|
@ -50,9 +50,16 @@ docker compose up -d
|
|||
```
|
||||
|
||||
this pulls `${READEST_IMAGE}` (default: `ghcr.io/readest/readest:latest`) instead of building the client locally.
|
||||
the web client now reads `SUPABASE_PUBLIC_URL`, `SUPABASE_ANON_KEY`, `API_BASE_URL`, `OBJECT_STORAGE_TYPE`, `STORAGE_FIXED_QUOTA`, and `TRANSLATION_FIXED_QUOTA` from runtime
|
||||
the web client now reads `SUPABASE_PUBLIC_URL`, `SUPABASE_ANON_KEY`, `API_BASE_URL`, `OBJECT_STORAGE_TYPE`, `SELF_HOSTED`, `STORAGE_FIXED_QUOTA`, and `TRANSLATION_FIXED_QUOTA` from runtime
|
||||
container env, so custom self-hosted values work with pulled images.
|
||||
|
||||
`SELF_HOSTED` unlocks every premium feature — third-party cloud sync (WebDAV,
|
||||
Google Drive, S3, OneDrive), offline Read Aloud downloads, Send to Readest —
|
||||
with or without a signed-in user. the image defaults it to `true`, so a pulled
|
||||
image is unlocked even if your `compose.yaml` predates the variable. set
|
||||
`SELF_HOSTED=false` in `docker/.env` only if you run a hosted service that sells
|
||||
the plans.
|
||||
|
||||
if you prefer Docker Hub, set `READEST_IMAGE` in `docker/.env`, for example:
|
||||
|
||||
```env
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue