diff --git a/.github/RELEASE_PROCESS.md b/.github/RELEASE_PROCESS.md index b37f3c3d..ca1e532a 100644 --- a/.github/RELEASE_PROCESS.md +++ b/.github/RELEASE_PROCESS.md @@ -4,15 +4,18 @@ Open Notebook uses a flow-driven release process. Work moves from `ready` issues into pull requests, pull requests merge to `main`, and maintainers cut a version when the branch has enough validated change to ship. +This document covers both the **mechanics** (how to cut, build and publish) and +the **confidence process** (how we know a release is good before users get it). +It was redesigned during the v1.11.0 release ([ADR-005](../docs/7-DEVELOPMENT/decisions/ADR-005-release-confidence-process.md)). + ## Release Model - Patch releases ship backwards-compatible fixes. - Minor releases ship backwards-compatible features and improvements. - Major releases are planned with a milestone when they include breaking changes or migrations that need user coordination. -- Release candidates and community soak labels are no longer part of the - process. Use the `in-dev-build` label for changes available in development - images and `released` for shipped work. +- Use the `in-dev-build` label for changes available in development images and + `released` for shipped work. ## Normal Flow @@ -21,59 +24,132 @@ version when the branch has enough validated change to ship. 3. Merge the pull request after review and required checks pass. 4. Let the development build publish the `v1-dev` image from `main`. 5. Cut a stable release when `main` has a coherent set of changes ready for - users. + users — following the confidence process below. + +## The Confidence Process + +Releases keep getting bigger; ad-hoc verification does not scale. Before +cutting, run this sequence: + +### 0. Changelog audit + +Diff `git log ..main` against the `[Unreleased]` section of the +CHANGELOG. Every merged PR must be represented (entries reference the issue +number when one exists, the PR number otherwise). The changelog is the input +for both the test plan and the release notes — close the gaps first, via PR. + +### 1. Risk-based test matrix + +Build a matrix from the actual release diff: each change → what it can break +and for whom → which bucket tests it. Pay special attention to +**"does the protection break legitimate use?"** for security changes (e.g. an +SSRF guard vs. self-hosted Ollama on localhost) and to anything a reverse +proxy, an upgrade, or a big upload would exercise. + +Buckets: + +- **A — automated, high confidence, run now**: full backend suite, frontend + lint/tests/production build, the smoke-e2e agent (full API happy path + UI + verification), targeted regression probes for the release's specific risks, + dependency audit. +- **B — automatable with investment**: decide per item whether to build the + muscle now (it compounds: the image gate below started as a bucket-B item) + or verify manually this once. +- **C — needs the release owner**: real provider credentials, real TTS podcast + generation, visual/UX judgment, and the final check of the pushed image. + +### 2. The image gate — test the artifact, not the repo + +A green suite on `main` is not a working image. Run: + +```bash +make docker-build-local # builds + local tags +make release-test TAG= OLD_TAG= +``` + +This runs two scenarios against real containers (`scripts/release-test/`): + +- **Fresh install**: empty DB → migrations on boot → in-image worker processes + a source → API/frontend/nginx-proxied checks. +- **Upgrade**: boot the *published* previous image, seed data, swap to the new + image on the same volume → migrations apply, data survives. + +Caveat: `docker-build-local` tags with the current `pyproject.toml` version — +`docker pull` the genuine previous tag before the upgrade test so you are not +comparing the new build against itself. + +### 3. Fix loop with a re-test policy + +Findings become focused PRs through the normal review flow. After each merge: +the cheap suite always re-runs; smoke/image gates re-run only if the fix +touches what they cover; manual verification is not repeated unless the fix +touches what was manually verified. Pre-existing bugs found along the way that +are not release regressions become backlog issues instead of scope creep. ## Cutting A Stable Release -1. Confirm `main` is green and review the changes since the previous release. -2. Update `pyproject.toml` with the target semantic version. -3. Move the relevant `CHANGELOG.md` entries from `Unreleased` into a dated - version section. -4. Create a GitHub release for the version tag, for example `v1.11.0`. -5. Run the `Build and Release` workflow. Push `v1-latest` tags for normal - stable releases. -6. Verify the published images are available in GHCR and Docker Hub when - Docker Hub credentials are configured. -7. Mark shipped issues with `released` and close any release-tracking tasks. +1. Confirm `main` is green and the confidence process above has run. +2. Open the **cut PR**: bump `pyproject.toml`, date the `[Unreleased]` section + as `[] - `. +3. After merge: `make tag`. +4. Build and push version images **via CI** (it holds the registry + credentials): trigger the *Build and Release* workflow with + `push_latest=false`. Local `make docker-push` also works but requires + `docker login` on both registries. +5. **Verify the pushed image** (bucket C, final gate): run it locally with + `make release-stack TAG= [DUMP=]` — a browsable, + isolated stack, optionally with a copy of real data — and walk the core + flows in the browser. +6. Publish the GitHub release. A non-prerelease publication triggers the + workflow again and pushes the `v1-latest` tags automatically. +7. Verify the `v1-latest` manifests on Docker Hub and GHCR (both arches, both + variants), and mark shipped issues with `released`. -## Docker Image Publishing +## Communication -The publishing workflow, from local test to promoted release: +Release notes follow this structure (see v1.11.0 as the reference): -```bash -make docker-build-local # 1. Build for the current platform, test locally -make docker-push # 2. Push version tags (does NOT update latest) - # 3. Test the pushed version in staging/production -make docker-push-latest # 4. Promote: push version + update v1-latest -``` +1. One-line verdict + upgrade recommendation. +2. Sections: Security, Features, Performance, Notable fixes. +3. **Behavior changes for self-hosters** — anything that can require a config + tweak on upgrade gets an explicit callout. +4. **Thanks** — credit every contributor by handle with what they shipped + (collect via `git log ..` + `gh pr view` for handles), plus + the issue reporters collectively. Never skip this section. + +Announce on Discord after `v1-latest` is live. + +## Retro + +Close every release by asking: what should improve in this process? Apply the +accepted improvements immediately — update this document, the scripts under +`scripts/release-test/`, and the decision log while the context is fresh. + +## Docker Image Publishing (reference) | Command | What it does | Updates latest? | |---------|--------------|-----------------| | `make docker-build-local` | Build for current platform only (tags `` + `local`) | No registry push | -| `make docker-push` | Push version tags to registries | ❌ No | -| `make docker-push-latest` | Push version + update `v1-latest` | ✅ Yes | -| `make docker-release` | Full release (same as docker-push-latest) | ✅ Yes | +| CI *Build and Release* (`push_latest=false`) | Push version tags via CI credentials | ❌ No | +| GitHub release published (non-prerelease) | CI pushes version + `v1-latest` | ✅ Yes | +| `make docker-push` / `docker-push-latest` | Local equivalents (need `docker login`) | ❌ / ✅ | | `make tag` | Create and push a git tag matching `pyproject.toml` | — | -Publishing details: - - **Platforms:** `linux/amd64`, `linux/arm64` - **Registries:** Docker Hub + GitHub Container Registry - **Image variants:** regular + single-container (`-single`) - **Version source:** `pyproject.toml` - Build issues: `docker builder prune`, then `make docker-buildx-reset` -## Manual Verification +## Known Gotchas -Before publishing a stable release, manually verify the areas touched by the -release. At minimum, cover: - -- installation or upgrade path changed by the release; -- source ingestion and processing when content or worker behavior changed; -- chat, search, notes, and notebooks when user workflows changed; -- provider setup when model, credential, or API behavior changed; -- Docker image startup when packaging, environment, or dependency changes - landed. - -Automated checks should catch regressions where possible, but the release owner -chooses the manual matrix from the actual changes in the release. +- **RC stack on non-default ports needs `API_URL`** or the browser talks to + `host:5055` — on a dev machine that is the development API (data crossover). + `rc-stack.sh` sets it; remember this for any custom setup. +- **Containerized app + host services**: credentials pointing at local + services (Ollama, LM Studio) need `http://host.docker.internal:`. +- **SurrealDB import**: `OVERWRITE` goes after the type keyword + (`DEFINE FIELD OVERWRITE …`), and the exporter can leak a log line into the + dump — `rc-stack.sh` handles both. +- **Multiple local SurrealDB instances**: check which one the dev `.env` + actually points at (`SURREAL_URL`) before exporting data. diff --git a/CHANGELOG.md b/CHANGELOG.md index 03b21f1d..c985936c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,6 +5,11 @@ All notable changes to this project will be documented in this file. The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). +## [Unreleased] + +### Added +- Release confidence process, documented and executable: `.github/RELEASE_PROCESS.md` now covers the risk-based test matrix (buckets A/B/C), the Docker image gate, the fix-loop re-test policy and the communication/credits/retro structure, backed by a new decision record (ADR-005) and versioned tooling under `scripts/release-test/` — `make release-test TAG= OLD_TAG=` runs fresh-install + upgrade scenarios against real images, and `make release-stack TAG= [DUMP=]` boots a browsable, isolated release-candidate stack (optionally with a copy of dev data) for manual verification + ## [1.11.0] - 2026-07-11 ### Added diff --git a/Makefile b/Makefile index 7a857532..e896f123 100644 --- a/Makefile +++ b/Makefile @@ -1,6 +1,7 @@ .PHONY: run frontend check ruff database lint api start-all stop-all status clean-cache worker worker-start worker-stop worker-restart .PHONY: docker-buildx-prepare docker-buildx-clean docker-buildx-reset .PHONY: docker-push docker-push-latest docker-release docker-build-local tag export-docs +.PHONY: release-test release-stack release-stack-down # Get version from pyproject.toml VERSION := $(shell grep -m1 version pyproject.toml | cut -d'"' -f2) @@ -43,6 +44,25 @@ docker-buildx-clean: docker-buildx-reset: docker-buildx-clean docker-buildx-prepare @echo "✅ Buildx reset complete!" +# === Release Testing (see .github/RELEASE_PROCESS.md) === + +# Automated image gate: fresh install + upgrade against real images. +# Usage: make release-test TAG=1.12.0 OLD_TAG=1.11.0 +release-test: + @test -n "$(TAG)" || (echo "usage: make release-test TAG= [OLD_TAG=]"; exit 1) + bash scripts/release-test/release-image-test.sh all \ + "$(DOCKERHUB_IMAGE):$(TAG)" \ + $(if $(OLD_TAG),"$(DOCKERHUB_IMAGE):$(OLD_TAG)") + +# Browsable RC stack for manual verification (optionally with a data dump). +# Usage: make release-stack TAG=1.12.0 [DUMP=/tmp/dev-dump.surql] +release-stack: + @test -n "$(TAG)" || (echo "usage: make release-stack TAG= [DUMP=]"; exit 1) + bash scripts/release-test/rc-stack.sh up "$(TAG)" $(DUMP) + +release-stack-down: + bash scripts/release-test/rc-stack.sh down "$(or $(TAG),unused)" + # === Docker Build Targets === # Build production image for local platform only (no push) diff --git a/docs/7-DEVELOPMENT/decisions/ADR-005-release-confidence-process.md b/docs/7-DEVELOPMENT/decisions/ADR-005-release-confidence-process.md new file mode 100644 index 00000000..6dfa983a --- /dev/null +++ b/docs/7-DEVELOPMENT/decisions/ADR-005-release-confidence-process.md @@ -0,0 +1,69 @@ +# ADR-005: Releases pass a risk-based confidence process, gated on the real image + +- **Status**: Accepted +- **Date**: 2026-07 (established during the v1.11.0 release) +- **Related**: [RELEASE_PROCESS.md](../../../.github/RELEASE_PROCESS.md), `scripts/release-test/`, [ADR-004](ADR-004-background-workers.md) + +## Context + +Releases grew from a handful of fixes to 50+ commits spanning security +hardening, features, migrations and dependency changes. Verification was +ad-hoc: a green test suite on `main` plus whatever manual checks the release +owner remembered. v1.11.0 proved the gap empirically — the unit suite was +fully green while `sort_by=title` returned a 500 (a SEARCH-index interaction +only a real SurrealDB exhibits) and clearing credential fields silently +no-oped (two mirror-image bugs, frontend and API, that only an end-to-end +path reveals). Neither class of bug is catchable by mocked tests, and neither +was: both were found by the process this record establishes. + +## Decision + +**Every stable release passes a risk-based confidence process before cutting, +and the final gate runs against the built Docker image — the artifact users +receive — not the repository.** + +The process (mechanics in [RELEASE_PROCESS.md](../../../.github/RELEASE_PROCESS.md)): + +1. **Changelog audit first** — the release diff, fully represented in the + CHANGELOG, is the input for both the test plan and the communication. +2. **Risk matrix over test list** — each change is classified by what it can + break and for whom, then assigned to a bucket: **A** (automated now), + **B** (automatable with investment — build the muscle when it compounds), + **C** (release-owner judgment: real credentials, real TTS, UX, the pushed + image). Security changes are probed for the inverse risk: does the + protection break legitimate use? +3. **The image gate** — fresh-install and upgrade-with-data scenarios run + against real containers (`make release-test`), because packaging bugs + (supervisord flags, uv sync modes, migration ordering) never appear in the + suite. +4. **Fix loop with a re-test policy** — findings become focused PRs; what + re-runs after each merge is defined up front. Pre-existing bugs that are + not release regressions become backlog issues, not scope creep. +5. **Human gates stay human** — the pushed-image verification and the release + publication require the release owner explicitly; automation prepares, + people pull the trigger. +6. **Retro closes the loop** — accepted improvements are applied to the + process docs and scripts in the same session. + +## Alternatives considered + +- **Keep ad-hoc verification** — free, but v1.11.0 showed it misses exactly + the bug classes that hurt users most (integration and packaging). +- **Full CI-based E2E on every PR** — highest coverage, but a real + SurrealDB + worker + image build pipeline on every PR is slow and expensive; + the release boundary is where artifact-level confidence pays off. +- **Community soak (RC tags)** — previously abandoned: slow feedback and low + participation; a deliberate confidence process front-loads what soaking + found late. + +## Consequences + +- Cutting a release costs hours, not minutes — deliberately: the cost scales + with release size, which is the point of the risk matrix. +- Release muscle is versioned in-repo (`scripts/release-test/`, make targets) + and compounds: bucket-B investments from one release become bucket-A + automation for the next. +- The upgrade scenario requires published previous images to remain available + on the registries. +- The process assumes a release owner in the loop for buckets B/C — it is a + confidence process, not full automation. diff --git a/scripts/release-test/docker-compose.release-test.yml b/scripts/release-test/docker-compose.release-test.yml new file mode 100644 index 00000000..7887b5f6 --- /dev/null +++ b/scripts/release-test/docker-compose.release-test.yml @@ -0,0 +1,43 @@ +services: + surrealdb: + image: surrealdb/surrealdb:v2 + command: start --log info --user root --pass root rocksdb:/mydata/test.db + user: root + volumes: + - ${DATA_DIR:?set DATA_DIR}/surreal:/mydata + + app: + image: ${APP_IMAGE:?set APP_IMAGE} + ports: + - "127.0.0.1:${API_PORT:?set API_PORT}:5055" + - "127.0.0.1:${FE_PORT:?set FE_PORT}:8502" + environment: + # Without an explicit API_URL the frontend's /config points the BROWSER + # at host:5055 — on a dev machine that is the development API, not this + # stack (silent data crossover between environments!) + - API_URL=${RC_API_URL:-} + - OPEN_NOTEBOOK_ENCRYPTION_KEY=${RC_ENCRYPTION_KEY:-release-test-key} + - SURREAL_URL=ws://surrealdb:8000/rpc + - SURREAL_USER=root + - SURREAL_PASSWORD=root + - SURREAL_NAMESPACE=${RC_SURREAL_NS:-open_notebook} + - SURREAL_DATABASE=${RC_SURREAL_DB:-open_notebook} + volumes: + - ${DATA_DIR}/notebook:/app/data + # Reach host services (e.g. Ollama on localhost:11434) via + # host.docker.internal on Linux too; Docker Desktop resolves it natively. + # Credentials pointing at local services must use + # http://host.docker.internal: when the app runs in a container. + extra_hosts: + - "host.docker.internal:host-gateway" + depends_on: + - surrealdb + + proxy: + image: nginx:alpine + ports: + - "127.0.0.1:${PROXY_PORT:?set PROXY_PORT}:80" + volumes: + - ./nginx.conf:/etc/nginx/conf.d/default.conf:ro + depends_on: + - app diff --git a/scripts/release-test/nginx.conf b/scripts/release-test/nginx.conf new file mode 100644 index 00000000..5c5c908e --- /dev/null +++ b/scripts/release-test/nginx.conf @@ -0,0 +1,18 @@ +# Reverse proxy in front of the frontend with default buffering ON — the SSE +# streaming fix must work anyway because the API sends X-Accel-Buffering: no, +# which nginx honors. Host is passed through verbatim to exercise the +# frontend's Host validation with a real proxy in the path. +server { + listen 80; + server_name _; + + location / { + proxy_pass http://app:8502; + proxy_http_version 1.1; + proxy_set_header Host $http_host; + proxy_set_header X-Forwarded-Proto $scheme; + proxy_set_header Upgrade $http_upgrade; + proxy_set_header Connection "upgrade"; + proxy_read_timeout 300s; + } +} diff --git a/scripts/release-test/rc-stack.sh b/scripts/release-test/rc-stack.sh new file mode 100755 index 00000000..0ef3d9b7 --- /dev/null +++ b/scripts/release-test/rc-stack.sh @@ -0,0 +1,75 @@ +#!/bin/bash +# Browsable release-candidate stack on the local machine: runs a pushed (or +# local) image, optionally with a COPY of your dev data, for manual release +# verification without touching the dev environment. +# +# Usage: +# rc-stack.sh up [dump.surql] # start (imports the dump if given) +# rc-stack.sh down # stop and remove everything +# +# To produce a data copy from a running dev SurrealDB (originals untouched): +# docker exec /surreal export \ +# --conn http://localhost:8000 --user root --pass root \ +# --ns open_notebook --db /dev/stdout > /tmp/dev-dump.surql +# +# Ports: UI http://localhost:18502 · via nginx http://localhost:18080 · API 15055 +set -euo pipefail +DIR="$(cd "$(dirname "$0")" && pwd)" +REPO="$(cd "$DIR/../.." && pwd)" +TAG="${2:?usage: rc-stack.sh [dump.surql]}" +DUMP="${3:-}" +RC_DATA=/tmp/onrel-rc-data + +# Reuse the dev encryption key so copied credentials decrypt; the DB name must +# match the one the dump came from (defaults to the dev .env's database). +KEY=$(grep '^OPEN_NOTEBOOK_ENCRYPTION_KEY' "$REPO/.env" 2>/dev/null | cut -d= -f2- | tr -d '"' || true) +DB=$(grep '^SURREAL_DATABASE' "$REPO/.env" 2>/dev/null | cut -d= -f2- | tr -d '"' || true) + +compose() { + APP_IMAGE="lfnovo/open_notebook:$TAG" DATA_DIR="$RC_DATA" \ + API_PORT=15055 FE_PORT=18502 PROXY_PORT=18080 \ + RC_API_URL="http://localhost:15055" \ + RC_ENCRYPTION_KEY="${KEY:-release-test-key}" RC_SURREAL_DB="${DB:-open_notebook}" \ + docker compose -p onrelrc -f "$DIR/docker-compose.release-test.yml" "$@" +} + +case "$1" in + down) + compose down -v + rm -rf "$RC_DATA" + echo "RC stack removed." + ;; + up) + compose down -v >/dev/null 2>&1 || true + rm -rf "$RC_DATA"; mkdir -p "$RC_DATA/surreal" "$RC_DATA/notebook" + if [ -n "$DUMP" ]; then + # SurrealDB import quirks, learned in production: + # - OVERWRITE goes AFTER the type keyword (DEFINE FIELD OVERWRITE ...), + # needed because RELATION tables auto-define in/out fields + # - the exporter can leak its own log line into the dump (starts with ESC) + sed -E $'/^\x1b/d; s/^DEFINE (TABLE|FIELD|INDEX|ANALYZER|FUNCTION|EVENT|PARAM|ACCESS) /DEFINE \\1 OVERWRITE /' "$DUMP" > "$RC_DATA/surreal/dump.surql" + fi + compose up -d surrealdb + sleep 4 + if [ -n "$DUMP" ]; then + docker exec onrelrc-surrealdb-1 /surreal import --conn http://localhost:8000 \ + --user root --pass root --ns open_notebook --db "${DB:-open_notebook}" \ + /mydata/dump.surql + fi + compose up -d + echo "Waiting for API..." + for i in $(seq 1 30); do + curl -sf -m 5 -o /dev/null http://localhost:15055/docs && break + sleep 5 + done + NB=$(curl -s http://localhost:15055/api/notebooks | python3 -c "import json,sys; print(len(json.load(sys.stdin)))" 2>/dev/null || echo "?") + echo "RC stack up — image lfnovo/open_notebook:$TAG" + echo " UI: http://localhost:18502" + echo " via nginx: http://localhost:18080" + echo " API: http://localhost:15055" + echo " notebooks: $NB" + echo + echo "NOTE: credentials pointing at host services (e.g. Ollama) need" + echo " base_url http://host.docker.internal: inside the container." + ;; +esac diff --git a/scripts/release-test/release-image-test.sh b/scripts/release-test/release-image-test.sh new file mode 100755 index 00000000..95642896 --- /dev/null +++ b/scripts/release-test/release-image-test.sh @@ -0,0 +1,150 @@ +#!/bin/bash +# Release image gate: fresh-install and upgrade tests against built Docker images. +# +# Usage: release-image-test.sh [old-image] +# e.g. release-image-test.sh all lfnovo/open_notebook:1.12.0 lfnovo/open_notebook:1.11.0 +# +# IMPORTANT: `make docker-build-local` tags the build with the CURRENT pyproject +# version. If that version matches a published release, `docker pull` the +# genuine old tag first or the upgrade test will compare the new build against +# itself. +set -u +DIR="$(cd "$(dirname "$0")" && pwd)" +COMPOSE="$DIR/docker-compose.release-test.yml" +NEW_IMAGE="${2:?usage: release-image-test.sh [old-image]}" +OLD_IMAGE="${3:-}" +PASS=0; FAIL=0 + +ok() { echo " ✅ $1"; PASS=$((PASS+1)); } +bad() { echo " ❌ $1"; FAIL=$((FAIL+1)); } +check() { if [ "$2" = "$3" ]; then ok "$1"; else bad "$1 (expected=$2, got=$3)"; fi; } + +# Each phase gets its own ports so a leaked container can never answer for the +# wrong stack (learned the hard way: a leftover fresh-install app once served +# the whole upgrade test). +set_ports() { API_PORT=$1; FE_PORT=$2; PROXY_PORT=$3; API="http://localhost:$1"; FE="http://localhost:$2"; PROXY="http://localhost:$3"; } + +compose_env() { env APP_IMAGE="$1" DATA_DIR="$2" API_PORT="$API_PORT" FE_PORT="$FE_PORT" PROXY_PORT="$PROXY_PORT" docker compose -p "$3" -f "$COMPOSE" "${@:4}"; } + +compose_up() { # + local OUT + OUT=$(compose_env "$1" "$2" "$3" up -d --quiet-pull 2>&1) + if echo "$OUT" | grep -qi "error"; then + bad "compose up ($3): $(echo "$OUT" | grep -i error | head -1)" + return 1 + fi + local RUNNING + RUNNING=$(docker inspect "$3-app-1" --format '{{.State.Running}}' 2>/dev/null) + check "container $3-app-1 running" "true" "$RUNNING" +} + +compose_down() { # + compose_env unused /tmp/unused "$1" down -v >/dev/null 2>&1 + if docker ps --format '{{.Names}}' | grep -q "^$1-"; then + bad "teardown of $1 left containers running" + fi + [ -n "${2:-}" ] && rm -rf "$2" +} + +wait_api() { # up to 150s (startup has ~50s of DB-wait backoff + migrations) + for i in $(seq 1 30); do + if curl -sf -m 5 -o /dev/null "$API/docs"; then return 0; fi + sleep 5 + done + return 1 +} + +seed_and_verify() { + NB=$(curl -s -X POST "$API/api/notebooks" -H "Content-Type: application/json" \ + -d '{"name":"release-probe","description":"release test seed"}' | python3 -c "import json,sys; print(json.load(sys.stdin)['id'])" 2>/dev/null) + [ -z "$NB" ] && { bad "create notebook"; return 1; } + ok "create notebook ($NB)" + SRC=$(curl -s -X POST "$API/api/sources" -F "type=text" -F "notebooks=[\"$NB\"]" \ + -F "content=Release test content. The Turing test evaluates machine intelligence." \ + -F "title=release-probe-source" -F "async_processing=true" -F "embed=false" \ + | python3 -c "import json,sys; print(json.load(sys.stdin)['id'])" 2>/dev/null) + [ -z "$SRC" ] && { bad "create source"; return 1; } + ok "create source ($SRC)" + ST="" + for i in $(seq 1 24); do + ST=$(curl -s "$API/api/sources/$SRC/status" | python3 -c "import json,sys; print(json.load(sys.stdin).get('status',''))" 2>/dev/null) + [ "$ST" = "completed" ] && break + sleep 5 + done + check "in-image worker processed source" "completed" "$ST" + FT=$(curl -s "$API/api/sources/$SRC" | python3 -c "import json,sys; d=json.load(sys.stdin); print('yes' if d.get('full_text') else 'no')" 2>/dev/null) + check "full_text present" "yes" "$FT" +} + +fresh_test() { + echo "═══ FRESH INSTALL — $NEW_IMAGE" + set_ports 15055 18502 18080 + local DD; DD=$(mktemp -d /tmp/onrel-fresh-XXXX) + compose_up "$NEW_IMAGE" "$DD" onrelfresh || { compose_down onrelfresh "$DD"; return 1; } + if wait_api; then ok "API up (migrations ran on boot)"; else + bad "API did not come up in 150s"; docker logs onrelfresh-app-1 2>&1 | tail -20 + compose_down onrelfresh "$DD"; return 1 + fi + N=$(curl -s "$API/api/notebooks" | python3 -c "import json,sys; print(len(json.load(sys.stdin)))" 2>/dev/null) + check "GET /api/notebooks on a virgin database" "0" "$N" + seed_and_verify + for f in type title created updated insights_count embedded; do + S=$(curl -s -o /dev/null -w '%{http_code}' "$API/api/sources?sort_by=$f&limit=2") + check "sort_by=$f" "200" "$S" + done + S=$(curl -s -o /dev/null -w '%{http_code}' -m 15 "$FE") + if [ "$S" = "200" ] || [ "$S" = "307" ]; then ok "frontend responds ($S)"; else bad "frontend ($S)"; fi + C=$(curl -s -m 10 "$PROXY/config") + echo "$C" | grep -q apiUrl && ok "/config via nginx: $C" || bad "/config via nginx: $C" + # Malformed Host: nginx may reject with 400 (correct) or the app falls back + # (200). What must never happen is a 5xx. + C=$(curl -s -m 10 -H 'Host: bad!' "$PROXY/config" -o /dev/null -w '%{http_code}') + if [ "$C" = "200" ] || [ "$C" = "400" ]; then ok "/config via nginx with malformed Host -> $C (no 5xx)"; else bad "/config malformed Host -> $C"; fi + compose_down onrelfresh "$DD" + echo +} + +upgrade_test() { + if [ -z "$OLD_IMAGE" ]; then echo "═══ UPGRADE skipped: no old image given"; return 0; fi + echo "═══ UPGRADE — $OLD_IMAGE -> $NEW_IMAGE" + set_ports 25055 28502 28080 + local DD; DD=$(mktemp -d /tmp/onrel-upg-XXXX) + echo "-- phase 1: boot $OLD_IMAGE and seed" + compose_up "$OLD_IMAGE" "$DD" onrelupg || { compose_down onrelupg "$DD"; return 1; } + IMG=$(docker inspect onrelupg-app-1 --format '{{.Config.Image}}' 2>/dev/null) + check "phase 1 runs the old image" "$OLD_IMAGE" "$IMG" + if wait_api; then ok "old image up"; else + bad "old image did not come up"; compose_down onrelupg "$DD"; return 1 + fi + seed_and_verify + echo "-- phase 2: swap to the new image on the SAME volume" + compose_env unused /tmp/unused onrelupg stop app >/dev/null 2>&1 + compose_env unused /tmp/unused onrelupg rm -f app >/dev/null 2>&1 + compose_up "$NEW_IMAGE" "$DD" onrelupg || { compose_down onrelupg "$DD"; return 1; } + IMG=$(docker inspect onrelupg-app-1 --format '{{.Config.Image}}' 2>/dev/null) + check "phase 2 runs the new image" "$NEW_IMAGE" "$IMG" + if wait_api; then ok "new image up on old data"; else + bad "new image did not come up on old data"; docker logs onrelupg-app-1 2>&1 | tail -20 + compose_down onrelupg "$DD"; return 1 + fi + FOUND=$(curl -s "$API/api/notebooks" | python3 -c " +import json,sys +nbs=json.load(sys.stdin) +print('yes' if any(n.get('name')=='release-probe' for n in nbs) else 'no')" 2>/dev/null) + check "notebook seeded on old image survived the upgrade" "yes" "$FOUND" + S=$(curl -s -o /dev/null -w '%{http_code}' "$API/api/sources?sort_by=title&limit=2") + check "sort_by=title after upgrade" "200" "$S" + echo " phase 2 migration log:" + docker logs onrelupg-app-1 2>&1 | grep -i "migrat" | tail -5 | sed 's/^/ /' + compose_down onrelupg "$DD" + echo +} + +case "${1:-all}" in + fresh) fresh_test ;; + upgrade) upgrade_test ;; + all) fresh_test; upgrade_test ;; +esac + +echo "═══ RESULT: $PASS passed, $FAIL failed" +[ "$FAIL" -eq 0 ]