mirror of
https://github.com/bytedance/deer-flow.git
synced 2026-07-09 16:08:31 +00:00
bc9ee9645c
Some checks are pending
Backend Blocking IO / backend-blocking-io (push) Waiting to run
Unit Tests / backend-unit-tests (push) Waiting to run
Frontend Unit Tests / frontend-unit-tests (push) Waiting to run
Lint Check / lint-backend (push) Waiting to run
Lint Check / lint-frontend (push) Waiting to run
Replay E2E (front-back contract) / Layer 1 — backend golden (no API key) (push) Waiting to run
Replay E2E (front-back contract) / Layer 2 — full-stack render (no API key) (push) Waiting to run
* feat(helm): add production-ready Helm chart for Kubernetes deployment Adds deploy/helm/deer-flow, a native-Kubernetes translation of the production docker-compose stack, plus CI to publish its images and chart. * ci(release): gate releases on version-source consistency Add a reusable verify-versions workflow invoked by both chart.yaml and container.yaml on v* tags. It runs scripts/verify_versions.sh against the tag and fails the release — skipping all image and chart publishing — when Chart.yaml (version + appVersion), backend/pyproject.toml, or frontend/package.json don't all match the tag. Add scripts/verify_versions.sh (the check, also runnable locally) and scripts/bump_version.sh (bumps all four sources in lockstep, then self-verifies). Document the release flow in RELEASING.md and link it from AGENTS.md. * fix(deploy): address Helm chart review feedback (#3987) Three review items from willem-bd: 1. nginx IPv6 listen strip never matched. The sed pattern required a `;` immediately after `2026`, but the rendered config emits `listen [::]:2026 default_server;` (space + `default_server` before the `;`), so the line was never deleted and nginx crash-looped on pods without IPv6 (`socket() :::2026 failed (97: Address family not supported)`). Drop the trailing `;` from the pattern so it matches. Same latent bug fixed in docker-compose-dev.yaml. 2. Passwords were spliced into DSNs verbatim, so a password containing URL-special chars (@ : / # ? % [ ] space) produced a malformed DSN and a confusing parse error. Add a `deer-flow.urlEscape` helper (replace-based: Sprig lacks urlqueryescape, and regexReplaceAllLiteral treats the replacement as a regex template so `[`/`]`/`?` break it) and apply it to the password in the postgres and redis DSNs. The raw `postgres-password` / `redis-password` keys stay unencoded - they back POSTGRES_PASSWORD / REDIS_PASSWORD, not a URL segment. 3. NODE_HOST defaulted to "gateway", which can never route: the gateway Service is ClusterIP:8001 and knows nothing of a sandbox NodePort, so a user who skips the caveat gets unreachable sandboxes with no error at install time. Default NODE_HOST to the provisioner pod's node IP via the downward API (status.hostIP) - a NodePort is exposed on every node, so <node-IP>:<NodePort> routes from the gateway on most clusters. `provisioner.nodeHost` remains an override for CNIs/policies that block pod->node-IP traffic. Updated NOTES.txt, values.yaml, and the chart README. (#3929 remains the long-term fix - ClusterIP + cluster-DNS URL removes NODE_HOST and the NodePort exposure entirely.) Validated with helm lint, helm template (incl. a special-char password rendering the encoded DSNs), and a sed pattern-match check. * fix(deploy): address round-2 Helm chart review feedback (#3987) Three "Medium" items from willem-bd: 1. No helm lint / helm template gate before publish. A template regression ships as an immutable OCI artifact (GHCR won't overwrite --version), so gate packaging on `helm lint` + `helm template --include-crds` in chart.yaml before `helm package`. (ct lint / helm-unittest deferred.) 2. Action pinning inconsistent + PR body overstates it. SHA-pin actions/checkout (v6.0.3, df4cb1c0) and actions/attest-build-provenance (v2.4.0, e8998f94) across the publishing workflows (chart.yaml, container.yaml, verify-versions.yml), matching the existing docker/* SHA-pin pattern. Resolves the checkout @v4/@v6 mismatch and makes the "SHA-pinned actions" claim accurate. Other pre-existing workflows left untouched (out of scope for this PR). 3. Provisioner RBAC broader than needed. Dropped the unused update/patch verbs and the pods/exec + events rules from the provisioner Role - audited against docker/provisioner/app.py, which only calls get/create/delete on pods and get/list/create/delete on services. Fixed NOTES.txt to accurately describe the grant instead of understating it as "create Pods and Services". The remaining scope concern - verbs apply to all Pods in the namespace, not just sandbox Pods - is still deferred (RBAC can't scope by label; needs a dedicated namespace or admission control), now noted in NOTES.txt and README. Validated with helm lint + helm template (narrowed Role renders with exactly get/list/watch/create/delete). * feat(helm): enable sandbox+web tools out of the box The chart's default config loaded zero agent tools (config.tools empty -> "Total tools loaded: 0"), so a fresh install gave an agent that could do nothing useful. Add tool_groups + tools to the default config block: - web: web_search (ddg), web_fetch (jina), image_search - no API key - file:read: ls, read_file, glob, grep - file:write: write_file, str_replace - bash The file/bash tools run inside the AIO sandbox the chart already configures; the web tools need outbound internet from the gateway pod (swap backends or drop entries for air-gapped clusters - see config.example.yaml). Also bump config_version 15 -> 19 to match config.example.yaml (the chart had drifted behind). NOTES.txt and the README example updated to match. * ci(helm): add chart validation + config_version drift check on PR Extend the chart workflow with a PR-triggered validate-chart job that runs helm lint, helm template --include-crds, and a config_version drift check: it parses config_version from both config.example.yaml and the chart's values.yaml and fails the build (with a ::error:: naming the files to bump) if the chart is behind the example. This catches the kind of drift this PR is fixing - the chart sat at v15 while the example moved to v19 - before it can merge again. verify-versions and publish-chart stay tag-only; publish-chart now needs: [verify-versions, validate-chart]. validate-chart runs on both PRs and tag pushes: the tag arm is required because a job that `needs` a skipped job is itself skipped under the default success() check, so validate-chart must actually run on tag pushes or publish-chart would never fire. * Bump config version to 20
126 lines
4.4 KiB
Markdown
126 lines
4.4 KiB
Markdown
# Releasing DeerFlow
|
|
|
|
DeerFlow releases are **tag-driven**: pushing a `v*` git tag triggers the
|
|
publishing workflows. There is no separate release script that bumps versions —
|
|
the maintainer bumps the version sources, updates the changelog, commits, and
|
|
tags. The helper scripts below keep the version sources in lockstep, and CI
|
|
gates the release on them agreeing with the tag.
|
|
|
|
## Version sources
|
|
|
|
A release version must appear, identically, in four places:
|
|
|
|
| File | Field |
|
|
| -------------------------------------- | -------------------- |
|
|
| `backend/pyproject.toml` | `version = "X.Y.Z"` |
|
|
| `frontend/package.json` | `"version": "X.Y.Z"` |
|
|
| `deploy/helm/deer-flow/Chart.yaml` | `version: X.Y.Z` |
|
|
| `deploy/helm/deer-flow/Chart.yaml` | `appVersion: "X.Y.Z"`|
|
|
|
|
Plus the git tag `vX.Y.Z` itself, which is the canonical release identifier.
|
|
|
|
Container images are tagged from the git tag (not from these files), and the
|
|
Helm chart version is validated against the tag — so if any source lags the
|
|
tag, the release is blocked (see [Version gate](#version-gate)).
|
|
|
|
## Helper scripts
|
|
|
|
- `scripts/bump_version.sh <version>` — set all four fields at once, then
|
|
self-verify. Tolerates a leading `v` (e.g. `v2.2.0`).
|
|
```bash
|
|
scripts/bump_version.sh 2.2.0
|
|
```
|
|
- `scripts/verify_versions.sh [version]` — check that all sources agree. With
|
|
no argument it requires mutual equality; with an argument it requires every
|
|
source to equal it. Exits non-zero on mismatch. Run it locally before tagging
|
|
to catch drift early:
|
|
```bash
|
|
scripts/verify_versions.sh 2.2.0
|
|
```
|
|
|
|
## Release procedure
|
|
|
|
1. **Bump the version** across all sources:
|
|
```bash
|
|
scripts/bump_version.sh 2.2.0
|
|
```
|
|
2. **Update `CHANGELOG.md`**: rename the `## [Unreleased]` section to
|
|
`## [2.2.0] — YYYY-MM-DD` (note the em dash `—`), and add a link reference
|
|
at the bottom of the file:
|
|
```
|
|
[2.2.0]: https://github.com/bytedance/deer-flow/releases/tag/v2.2.0
|
|
```
|
|
Start a fresh `## [Unreleased]` section above it for the next cycle.
|
|
3. **Commit** the version + changelog changes:
|
|
```bash
|
|
git add -A
|
|
git commit -m "release: v2.2.0"
|
|
```
|
|
4. **Tag and push**:
|
|
```bash
|
|
git tag v2.2.0
|
|
git push origin v2.2.0
|
|
```
|
|
Pushing the tag triggers the publishing workflows (below).
|
|
|
|
## What CI publishes on a `v*` tag
|
|
|
|
- `.github/workflows/container.yaml` — builds and pushes `backend`,
|
|
`frontend`, and `provisioner` images to `ghcr.io`, tagged with the release
|
|
version (and `latest` on the default branch).
|
|
- `.github/workflows/chart.yaml` — packages the Helm chart and pushes it as an
|
|
OCI artifact to `ghcr.io`. Users install with:
|
|
```bash
|
|
helm install deer-flow oci://ghcr.io/<owner>/deer-flow --version 2.2.0
|
|
```
|
|
|
|
## Version gate
|
|
|
|
Both publishing workflows call `.github/workflows/verify-versions.yml` as their
|
|
first job. It runs `scripts/verify_versions.sh` against the tag (minus the
|
|
`v`). If any of the four version sources doesn't match the tag, the verify job
|
|
fails and **all** publish jobs are skipped — no images, no chart.
|
|
|
|
When it fails, the job annotation names the offending file and suggests the
|
|
fix:
|
|
|
|
```
|
|
::error::frontend/package.json is '2.1.0' but expected '2.2.0'.
|
|
Tip: run scripts/bump_version.sh 2.2.0 to align all sources.
|
|
```
|
|
|
|
## Pre-releases (RCs)
|
|
|
|
Pre-release tags like `v2.2.0-rc1` are valid `v*` tags and trigger the same
|
|
workflows. The version sources must equal the full pre-release string
|
|
(`2.2.0-rc1`) — the gate compares exact strings. Use the same procedure with
|
|
the rc version:
|
|
|
|
```bash
|
|
scripts/bump_version.sh 2.2.0-rc1
|
|
# update CHANGELOG, commit, tag v2.2.0-rc1, push
|
|
```
|
|
|
|
## Recovering from a failed gate
|
|
|
|
If the gate failed because a source was forgotten:
|
|
|
|
1. Run `scripts/bump_version.sh <version>` to align the sources.
|
|
2. Amend or add a follow-up commit.
|
|
3. Delete and re-create the tag, then push it:
|
|
```bash
|
|
git tag -d v2.2.0
|
|
git tag v2.2.0
|
|
git push origin :refs/tags/v2.2.0
|
|
git push origin v2.2.0
|
|
```
|
|
|
|
Re-pushing the tag re-triggers the workflows. Because the gate blocks **all**
|
|
artifacts when it fails, nothing was published under the bad tag, so re-tagging
|
|
is safe — no images or chart were pushed to overwrite.
|
|
|
|
## Post-release
|
|
|
|
Optionally draft a **GitHub Release** from the tag, pasting the corresponding
|
|
`CHANGELOG.md` section as the release notes. The changelog link references
|
|
point at these release URLs.
|