Commit graph

2 commits

Author SHA1 Message Date
morgankrey
2ec2b4c600
Fix and improve docs automation scripts (#50120)
## Summary

Fixes issues discovered while running the docs automation workflow for
the first time, plus improvements based on the v0.225 run where 44
suggestions overwhelmed a single Droid invocation.

### docs-suggest-publish
- Ignore untracked files when checking for clean working directory
- Add `--auto high` flag to droid exec for non-interactive use
- Add error handling to show droid output on failure
- Remove non-existent `documentation` label from PR creation
- Use `--write` flag for prettier to fix formatting
- **Batch suggestions** into groups of 10 (configurable with
`--batch-size`) to prevent Droid from dropping suggestions when context
is too large
- **Pre-PR docs build validation** — runs `generate-action-metadata` +
`mdbook build` before creating the PR to catch invalid `{#action}` and
`{#kb}` references locally instead of waiting for CI (skippable with
`--skip-validation`)
- **Prompt guardrail** — instructs Droid not to invent `{#kb}` or
`{#action}` references, only reusing action names already present in
docs files
- **Stable release detection** — at publish time, checks each queued
PR's merge commit against the latest stable release tag. PRs already in
stable get annotated "ALREADY IN STABLE" so Droid applies content
changes without adding incorrect Preview callouts
- **Feature flag detection** — parses
`crates/feature_flags/src/flags.rs` for all feature flag struct names,
then checks each PR's diff for references. PRs behind feature flags are
skipped entirely since those features aren't generally available yet

### docs-strip-preview-callouts
- Remove non-existent `documentation` label from PR creation
- Add `Release Notes: - N/A` to generated PR body (fixes Danger bot
check)

## Context

These scripts were run for the first time as part of the v0.225 release.
Issues found:
1. The `documentation` label doesn't exist in this repo
2. Droid exec needs `--auto high` for non-interactive execution
3. Prettier needs `--write` to actually fix files (was running in check
mode)
4. Untracked files should not block the workflow
5. Sending all 44 suggestions in one Droid invocation only applied 2 —
batching in groups of 10 fixed this
6. Droid hallucinated action names (`settings::OpenSettings`,
`gpui::Modifiers::secondary_key`) that broke the docs preprocessor build
7. PRs that shipped in stable v0.225 incorrectly got Preview callouts
because the queue doesn't distinguish preview-only from
already-in-stable
8. PRs behind feature flags (subagents, git graph) got documented
despite not being generally available

Release Notes:

- N/A
2026-02-25 19:02:37 -06:00
morgankrey
dc41f71f57
Add documentation suggestion automation (#49194)
Adds scripts and a GitHub Action workflow for automatically suggesting
documentation updates when PRs modify user-facing code.

## Scripts

- **`script/docs-suggest`**: Analyze PRs/commits for documentation needs
using AI
- **`script/docs-suggest-publish`**: Create a PR from batched
suggestions
- **`script/docs-strip-preview-callouts`**: Remove Preview callouts when
shipping to stable
- **`script/test-docs-suggest-batch`**: Testing utility for batch
analysis

## Workflow

The GitHub Action (`.github/workflows/docs_suggestions.yml`) handles two
scenarios:

1. **PRs merged to main**: Suggestions are batched to
`docs/suggestions-pending` branch for the next Preview release
2. **Cherry-picks to release branches**: Suggestions are posted as PR
comments for immediate review

## Callout Types

The system distinguishes between:

- **Additive features** (new commands, settings, UI):
  ```markdown
> **Preview:** This feature is available in Zed Preview. It will be
included in the next Stable release.
  ```

- **Behavior modifications** (changed defaults, altered existing
behavior):
  ```markdown
> **Changed in Preview (v0.XXX).** See [release notes](/releases#0.XXX).
  ```

Both callout types are stripped by `docs-strip-preview-callouts` when
features ship to stable.

## Example Output

See PR #49190 for example documentation suggestions generated by running
this on PRs from the v0.224 preview window.

## Usage

```bash
# Analyze a PR (auto-detects batch vs immediate mode)
script/docs-suggest --pr 49100

# Dry run to see assembled context
script/docs-suggest --pr 49100 --dry-run

# Create PR from batched suggestions
script/docs-suggest-publish

# Strip callouts for stable release
script/docs-strip-preview-callouts
```

Release Notes:

- N/A
2026-02-18 06:39:09 -06:00