Add documentation suggestion automation (#49194)

Adds scripts and a GitHub Action workflow for automatically suggesting
documentation updates when PRs modify user-facing code.

## Scripts

- **`script/docs-suggest`**: Analyze PRs/commits for documentation needs
using AI
- **`script/docs-suggest-publish`**: Create a PR from batched
suggestions
- **`script/docs-strip-preview-callouts`**: Remove Preview callouts when
shipping to stable
- **`script/test-docs-suggest-batch`**: Testing utility for batch
analysis

## Workflow

The GitHub Action (`.github/workflows/docs_suggestions.yml`) handles two
scenarios:

1. **PRs merged to main**: Suggestions are batched to
`docs/suggestions-pending` branch for the next Preview release
2. **Cherry-picks to release branches**: Suggestions are posted as PR
comments for immediate review

## Callout Types

The system distinguishes between:

- **Additive features** (new commands, settings, UI):
  ```markdown
> **Preview:** This feature is available in Zed Preview. It will be
included in the next Stable release.
  ```

- **Behavior modifications** (changed defaults, altered existing
behavior):
  ```markdown
> **Changed in Preview (v0.XXX).** See [release notes](/releases#0.XXX).
  ```

Both callout types are stripped by `docs-strip-preview-callouts` when
features ship to stable.

## Example Output

See PR #49190 for example documentation suggestions generated by running
this on PRs from the v0.224 preview window.

## Usage

```bash
# Analyze a PR (auto-detects batch vs immediate mode)
script/docs-suggest --pr 49100

# Dry run to see assembled context
script/docs-suggest --pr 49100 --dry-run

# Create PR from batched suggestions
script/docs-suggest-publish

# Strip callouts for stable release
script/docs-strip-preview-callouts
```

Release Notes:

- N/A
This commit is contained in:
morgankrey 2026-02-18 06:39:09 -06:00 committed by GitHub
parent d5d49c1717
commit dc41f71f57
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
6 changed files with 1749 additions and 0 deletions

358
.github/workflows/docs_suggestions.yml vendored Normal file
View file

@ -0,0 +1,358 @@
name: Documentation Suggestions
# Stable release callout stripping plan (not wired yet):
# 1. Add a separate stable-only workflow trigger on `release.published`
# with `github.event.release.prerelease == false`.
# 2. In that workflow, run `script/docs-strip-preview-callouts` on `main`.
# 3. Open a PR with stripped preview callouts for human review.
# 4. Fail loudly on script errors or when no callout changes are produced.
# 5. Keep this workflow focused on suggestions only until that stable workflow is added.
on:
# Run when PRs are merged to main
pull_request:
types: [closed]
branches: [main]
paths:
- 'crates/**/*.rs'
- '!crates/**/*_test.rs'
- '!crates/**/tests/**'
# Run on cherry-picks to release branches
pull_request_target:
types: [opened, synchronize]
branches:
- 'v0.*'
paths:
- 'crates/**/*.rs'
# Manual trigger for testing
workflow_dispatch:
inputs:
pr_number:
description: 'PR number to analyze'
required: true
type: string
mode:
description: 'Output mode'
required: true
type: choice
options:
- batch
- immediate
default: batch
permissions:
contents: write
pull-requests: write
env:
DROID_MODEL: claude-sonnet-4-5-20250929
SUGGESTIONS_BRANCH: docs/suggestions-pending
jobs:
# Job for PRs merged to main - batch suggestions to branch
batch-suggestions:
runs-on: ubuntu-latest
timeout-minutes: 10
if: |
(github.event_name == 'pull_request' &&
github.event.pull_request.merged == true &&
github.event.pull_request.base.ref == 'main') ||
(github.event_name == 'workflow_dispatch' && inputs.mode == 'batch')
steps:
- name: Checkout repository
uses: actions/checkout@v4
with:
fetch-depth: 0
token: ${{ secrets.GITHUB_TOKEN }}
- name: Install Droid CLI
run: |
curl -fsSL https://app.factory.ai/cli | sh
echo "${HOME}/.local/bin" >> "$GITHUB_PATH"
env:
FACTORY_API_KEY: ${{ secrets.FACTORY_API_KEY }}
- name: Get PR info
id: pr
run: |
if [ -n "${{ inputs.pr_number }}" ]; then
PR_NUM="${{ inputs.pr_number }}"
else
PR_NUM="${{ github.event.pull_request.number }}"
fi
echo "number=$PR_NUM" >> "$GITHUB_OUTPUT"
# Get PR title
PR_TITLE=$(gh pr view "$PR_NUM" --json title --jq '.title')
echo "title=$PR_TITLE" >> "$GITHUB_OUTPUT"
env:
GH_TOKEN: ${{ github.token }}
- name: Analyze PR for documentation needs
id: analyze
run: |
OUTPUT_FILE=$(mktemp)
./script/docs-suggest \
--pr "${{ steps.pr.outputs.number }}" \
--immediate \
--preview \
--output "$OUTPUT_FILE" \
--verbose
# Check if we got actionable suggestions (not "no updates needed")
if grep -q "Documentation Suggestions" "$OUTPUT_FILE" && \
! grep -q "No Documentation Updates Needed" "$OUTPUT_FILE"; then
echo "has_suggestions=true" >> "$GITHUB_OUTPUT"
echo "output_file=$OUTPUT_FILE" >> "$GITHUB_OUTPUT"
else
echo "has_suggestions=false" >> "$GITHUB_OUTPUT"
echo "No actionable documentation suggestions for this PR"
cat "$OUTPUT_FILE"
fi
env:
GH_TOKEN: ${{ github.token }}
FACTORY_API_KEY: ${{ secrets.FACTORY_API_KEY }}
- name: Commit suggestions to queue branch
if: steps.analyze.outputs.has_suggestions == 'true'
run: |
set -euo pipefail
PR_NUM="${{ steps.pr.outputs.number }}"
PR_TITLE="${{ steps.pr.outputs.title }}"
OUTPUT_FILE="${{ steps.analyze.outputs.output_file }}"
# Configure git
git config user.name "github-actions[bot]"
git config user.email "github-actions[bot]@users.noreply.github.com"
# Retry loop for handling concurrent pushes
MAX_RETRIES=3
for i in $(seq 1 "$MAX_RETRIES"); do
echo "Attempt $i of $MAX_RETRIES"
# Fetch and checkout suggestions branch (create if doesn't exist)
if git ls-remote --exit-code --heads origin "$SUGGESTIONS_BRANCH" > /dev/null 2>&1; then
git fetch origin "$SUGGESTIONS_BRANCH"
git checkout -B "$SUGGESTIONS_BRANCH" "origin/$SUGGESTIONS_BRANCH"
else
# Create orphan branch for clean history
git checkout --orphan "$SUGGESTIONS_BRANCH"
git rm -rf . > /dev/null 2>&1 || true
# Initialize with README
cat > README.md << 'EOF'
# Documentation Suggestions Queue
This branch contains batched documentation suggestions for the next Preview release.
Each file represents suggestions from a merged PR. At preview branch cut time,
run `script/docs-suggest-publish` to create a documentation PR from these suggestions.
## Structure
- `suggestions/PR-XXXXX.md` - Suggestions for PR #XXXXX
- `manifest.json` - Index of all pending suggestions
## Workflow
1. PRs merged to main trigger documentation analysis
2. Suggestions are committed here as individual files
3. At preview release, suggestions are collected into a docs PR
4. After docs PR is created, this branch is reset
EOF
mkdir -p suggestions
echo '{"suggestions":[]}' > manifest.json
git add README.md suggestions manifest.json
git commit -m "Initialize documentation suggestions queue"
fi
# Create suggestion file
SUGGESTION_FILE="suggestions/PR-${PR_NUM}.md"
{
echo "# PR #${PR_NUM}: ${PR_TITLE}"
echo ""
echo "_Merged: $(date -u +%Y-%m-%dT%H:%M:%SZ)_"
echo "_PR: https://github.com/${{ github.repository }}/pull/${PR_NUM}_"
echo ""
cat "$OUTPUT_FILE"
} > "$SUGGESTION_FILE"
# Update manifest
MANIFEST=$(cat manifest.json)
NEW_ENTRY="{\"pr\":${PR_NUM},\"title\":$(echo "$PR_TITLE" | jq -R .),\"file\":\"$SUGGESTION_FILE\",\"date\":\"$(date -u +%Y-%m-%dT%H:%M:%SZ)\"}"
# Add to manifest if not already present
if ! echo "$MANIFEST" | jq -e ".suggestions[] | select(.pr == $PR_NUM)" > /dev/null 2>&1; then
echo "$MANIFEST" | jq ".suggestions += [$NEW_ENTRY]" > manifest.json
fi
# Commit
git add "$SUGGESTION_FILE" manifest.json
git commit -m "docs: Add suggestions for PR #${PR_NUM}
${PR_TITLE}
Auto-generated documentation suggestions for review at next preview release."
# Try to push
if git push origin "$SUGGESTIONS_BRANCH"; then
echo "Successfully pushed suggestions"
break
else
echo "Push failed, retrying..."
if [ "$i" -eq "$MAX_RETRIES" ]; then
echo "Failed after $MAX_RETRIES attempts"
exit 1
fi
sleep $((i * 2))
fi
done
- name: Summary
if: always()
run: |
{
echo "## Documentation Suggestions"
echo ""
if [ "${{ steps.analyze.outputs.has_suggestions }}" == "true" ]; then
echo "✅ Suggestions queued for PR #${{ steps.pr.outputs.number }}"
echo ""
echo "View pending suggestions: [docs/suggestions-pending branch](https://github.com/${{ github.repository }}/tree/${{ env.SUGGESTIONS_BRANCH }})"
else
echo "No documentation updates needed for this PR."
fi
} >> "$GITHUB_STEP_SUMMARY"
# Job for cherry-picks to release branches - immediate output to step summary
cherry-pick-suggestions:
runs-on: ubuntu-latest
timeout-minutes: 10
if: |
(github.event_name == 'pull_request_target' &&
startsWith(github.event.pull_request.base.ref, 'v0.')) ||
(github.event_name == 'workflow_dispatch' && inputs.mode == 'immediate')
steps:
- name: Checkout repository
uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Install Droid CLI
run: |
curl -fsSL https://app.factory.ai/cli | sh
echo "${HOME}/.local/bin" >> "$GITHUB_PATH"
env:
FACTORY_API_KEY: ${{ secrets.FACTORY_API_KEY }}
- name: Get PR number
id: pr
run: |
if [ -n "${{ inputs.pr_number }}" ]; then
echo "number=${{ inputs.pr_number }}" >> "$GITHUB_OUTPUT"
else
echo "number=${{ github.event.pull_request.number }}" >> "$GITHUB_OUTPUT"
fi
- name: Analyze PR for documentation needs
id: analyze
run: |
OUTPUT_FILE=$(mktemp)
# Cherry-picks don't get preview callout
./script/docs-suggest \
--pr "${{ steps.pr.outputs.number }}" \
--immediate \
--no-preview \
--output "$OUTPUT_FILE" \
--verbose
# Check if we got actionable suggestions
if [ -s "$OUTPUT_FILE" ] && \
grep -q "Documentation Suggestions" "$OUTPUT_FILE" && \
! grep -q "No Documentation Updates Needed" "$OUTPUT_FILE"; then
echo "has_suggestions=true" >> "$GITHUB_OUTPUT"
{
echo 'suggestions<<EOF'
cat "$OUTPUT_FILE"
echo 'EOF'
} >> "$GITHUB_OUTPUT"
else
echo "has_suggestions=false" >> "$GITHUB_OUTPUT"
fi
env:
GH_TOKEN: ${{ github.token }}
FACTORY_API_KEY: ${{ secrets.FACTORY_API_KEY }}
- name: Post suggestions as PR comment
if: steps.analyze.outputs.has_suggestions == 'true'
uses: actions/github-script@v7
with:
script: |
const suggestions = `${{ steps.analyze.outputs.suggestions }}`;
const body = `## 📚 Documentation Suggestions
This cherry-pick contains changes that may need documentation updates.
${suggestions}
---
<details>
<summary>About this comment</summary>
This comment was generated automatically by analyzing code changes in this cherry-pick.
Cherry-picks typically don't need new documentation since the feature was already
documented when merged to main, but please verify.
</details>`;
// Find existing comment to update (avoid spam)
const { data: comments } = await github.rest.issues.listComments({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: ${{ steps.pr.outputs.number }}
});
const botComment = comments.find(c =>
c.user.type === 'Bot' &&
c.body.includes('Documentation Suggestions')
);
if (botComment) {
await github.rest.issues.updateComment({
owner: context.repo.owner,
repo: context.repo.repo,
comment_id: botComment.id,
body: body
});
} else {
await github.rest.issues.createComment({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: ${{ steps.pr.outputs.number }},
body: body
});
}
- name: Summary
if: always()
run: |
{
echo "## 📚 Documentation Suggestions (Cherry-pick)"
echo ""
if [ "${{ steps.analyze.outputs.has_suggestions }}" == "true" ]; then
echo "Suggestions posted as PR comment."
echo ""
echo "${{ steps.analyze.outputs.suggestions }}"
else
echo "No documentation suggestions for this cherry-pick."
fi
} >> "$GITHUB_STEP_SUMMARY"

View file

@ -80,6 +80,18 @@ description: One sentence describing what this page covers. Used in search resul
- `title`: Feature name, optionally with "- Zed" suffix for SEO
- `description`: Concise summary for search engines and link previews
- Keep frontmatter values as simple single-line `key: value` entries (no
multiline values, no quotes) for compatibility with the docs postprocessor
#### Frontmatter SEO Guidelines
- Choose one primary keyword/intent phrase for each page
- Write unique `title` values that clearly state the page topic and target user
intent; aim for ~50-60 characters
- Write `description` values that summarize what the reader can do on the page;
aim for ~140-160 characters
- Use the primary keyword naturally in the `title` and page body at least once
(usually in the opening paragraph); avoid keyword stuffing
### Section Ordering
@ -235,6 +247,20 @@ End pages with related links when helpful:
- [Inline Assistant](./inline-assistant.md): Prompt-driven code transformations
```
### SEO Linking Guidelines
- Ensure each page is reachable from at least one other docs page (no orphan
pages)
- For non-reference pages, include at least 3 internal links to related docs
when possible
- Reference pages (for example, `docs/src/reference/*`) can use fewer links when
extra links would add noise
- Add links to closely related docs where they help users complete the next task
- Use descriptive link text that tells users what they will get on the linked
page
- For main feature pages with a matching marketing page, include a relevant
`zed.dev` marketing link in addition to docs links
---
## Language-Specific Documentation
@ -327,6 +353,8 @@ Before any documentation change is considered complete:
Before finalizing documentation:
- [ ] Frontmatter includes `title` and `description`
- [ ] Page has a clear primary keyword/intent phrase
- [ ] Primary keyword appears naturally in the page body (no keyword stuffing)
- [ ] Opening paragraph explains what and why
- [ ] Settings show UI first, then JSON examples
- [ ] Actions use `{#action ...}` and `{#kb ...}` syntax
@ -334,6 +362,8 @@ Before finalizing documentation:
- [ ] Anchor IDs on sections likely to be linked
- [ ] Version callouts where behavior differs by release
- [ ] No orphan pages (linked from somewhere)
- [ ] Non-reference pages include at least 3 useful internal docs links
- [ ] Main feature pages include a relevant `zed.dev` marketing link
- [ ] Passes Prettier formatting check
- [ ] Passes brand voice rubric (see `brand-voice/rubric.md`)

View file

@ -0,0 +1,228 @@
#!/usr/bin/env bash
#
# Remove Preview callouts from documentation for stable release.
#
# Usage:
# script/docs-strip-preview-callouts [--dry-run]
#
# This script finds and removes all Preview-related callouts from docs:
# > **Preview:** This feature is available in Zed Preview...
# > **Changed in Preview (v0.XXX).** See [release notes]...
#
# Then creates a PR with the changes.
#
# Options:
# --dry-run Show what would be changed without modifying files or creating PR
# --verbose Show detailed progress
#
# Run this as part of the stable release workflow.
set -euo pipefail
DRY_RUN=false
VERBOSE=false
# Colors
RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[0;33m'
BLUE='\033[0;34m'
NC='\033[0m'
log() {
if [[ "$VERBOSE" == "true" ]]; then
echo -e "${BLUE}[strip-preview]${NC} $*" >&2
fi
}
error() {
echo -e "${RED}Error:${NC} $*" >&2
exit 1
}
# Parse arguments
while [[ $# -gt 0 ]]; do
case $1 in
--dry-run)
DRY_RUN=true
shift
;;
--verbose)
VERBOSE=true
shift
;;
-h|--help)
head -18 "$0" | tail -16
exit 0
;;
*)
error "Unknown option: $1"
;;
esac
done
# Get repo root
REPO_ROOT="$(cd "$(dirname "$0")/.." && pwd)"
cd "$REPO_ROOT"
DOCS_DIR="$REPO_ROOT/docs/src"
echo "Searching for Preview callouts in $DOCS_DIR..."
# Find files with either type of preview callout:
# - > **Preview:** ...
# - > **Changed in Preview ...
files_with_callouts=$(grep -rlE "> \*\*(Preview:|Changed in Preview)" "$DOCS_DIR" 2>/dev/null || true)
if [[ -z "$files_with_callouts" ]]; then
echo "No Preview callouts found. Nothing to do."
exit 0
fi
file_count=$(echo "$files_with_callouts" | wc -l | tr -d ' ')
echo "Found $file_count file(s) with Preview callouts:"
echo ""
for file in $files_with_callouts; do
relative_path="${file#$REPO_ROOT/}"
echo " $relative_path"
if [[ "$VERBOSE" == "true" ]]; then
grep -nE "> \*\*(Preview:|Changed in Preview)" "$file" | while read -r line; do
echo " $line"
done
fi
done
echo ""
if [[ "$DRY_RUN" == "true" ]]; then
echo -e "${YELLOW}=== DRY RUN ===${NC}"
echo ""
echo "Would remove Preview callouts from the files above and create a PR."
echo ""
echo "Lines to be removed:"
echo ""
for file in $files_with_callouts; do
relative_path="${file#$REPO_ROOT/}"
echo "--- $relative_path ---"
grep -nE "> \*\*(Preview:|Changed in Preview)" "$file" || true
echo ""
done
echo -e "${YELLOW}=== END DRY RUN ===${NC}"
echo ""
echo "Run without --dry-run to apply changes and create PR."
exit 0
fi
# Check for clean working state
if [[ -n "$(git status --porcelain docs/)" ]]; then
error "docs/ directory has uncommitted changes. Please commit or stash first."
fi
# Apply changes
echo "Removing Preview callouts..."
for file in $files_with_callouts; do
log "Processing: $file"
tmp_file=$(mktemp)
# Remove preview callout lines and their continuations
# Handles both:
# > **Preview:** This feature is available...
# > **Changed in Preview (v0.XXX).** See [release notes]...
awk '
BEGIN { in_callout = 0 }
/^> \*\*Preview:\*\*/ {
in_callout = 1
next
}
/^> \*\*Changed in Preview/ {
in_callout = 1
next
}
in_callout && /^>/ && !/^> \*\*/ {
next
}
in_callout && /^$/ {
in_callout = 0
next
}
{
in_callout = 0
print
}
' "$file" > "$tmp_file"
mv "$tmp_file" "$file"
echo " Updated: ${file#$REPO_ROOT/}"
done
echo ""
echo -e "${GREEN}Preview callouts removed from $file_count file(s).${NC}"
# Check if there are actual changes (in case callouts were in comments or something)
if [[ -z "$(git status --porcelain docs/)" ]]; then
echo ""
echo "No effective changes to commit (callouts may have been in non-rendered content)."
exit 0
fi
# Create branch and PR
echo ""
echo "Creating PR..."
BRANCH_NAME="docs/stable-release-$(date +%Y-%m-%d)"
log "Branch: $BRANCH_NAME"
git checkout -b "$BRANCH_NAME"
git add docs/src/
# Build file list for commit message
FILE_LIST=$(echo "$files_with_callouts" | sed "s|$REPO_ROOT/||" | sed 's/^/- /')
git commit -m "docs: Remove Preview callouts for stable release
Features documented with Preview callouts are now in Stable.
Files updated:
$FILE_LIST"
git push -u origin "$BRANCH_NAME"
gh pr create \
--title "docs: Remove Preview callouts for stable release" \
--body "This PR removes Preview callouts from documentation for features that are now in Stable.
## Files Updated
$(echo "$files_with_callouts" | sed "s|$REPO_ROOT/|• |")
## What This Does
Removes callouts like:
\`\`\`markdown
> **Preview:** This feature is available in Zed Preview. It will be included in the next Stable release.
\`\`\`
And:
\`\`\`markdown
> **Changed in Preview (v0.XXX).** See [release notes](/releases#0.XXX).
\`\`\`
These features are now in Stable, so the callouts are no longer needed." \
--label "documentation"
PR_URL=$(gh pr view --json url --jq '.url')
echo ""
echo -e "${GREEN}Done!${NC}"
echo ""
echo "PR created: $PR_URL"
echo ""
echo "Next steps:"
echo "1. Review the PR to ensure callouts were removed correctly"
echo "2. Merge the PR as part of the stable release"

616
script/docs-suggest Executable file
View file

@ -0,0 +1,616 @@
#!/usr/bin/env bash
#
# Analyze code changes and suggest documentation updates.
#
# Usage:
# script/docs-suggest --pr 49177 # Analyze a PR (immediate mode)
# script/docs-suggest --commit abc123 # Analyze a commit
# script/docs-suggest --diff file.patch # Analyze a diff file
# script/docs-suggest --staged # Analyze staged changes
#
# Modes:
# --batch Append suggestions to batch file for later PR (default for main)
# --immediate Output suggestions directly (default for cherry-picks)
#
# Options:
# --dry-run Show assembled context without calling LLM
# --output FILE Write suggestions to file instead of stdout (immediate mode)
# --verbose Show detailed progress
# --model NAME Override default model
# --preview Add preview callout to suggested docs (auto-detected)
#
# Batch mode:
# Suggestions are appended to docs/.suggestions/pending.md
# Use script/docs-suggest-publish to create a PR from batched suggestions
#
# Examples:
# # Analyze a PR to main (batches by default)
# script/docs-suggest --pr 49100
#
# # Analyze a cherry-pick PR (immediate by default)
# script/docs-suggest --pr 49152
#
# # Force immediate output for testing
# script/docs-suggest --pr 49100 --immediate
#
# # Dry run to see context
# script/docs-suggest --pr 49100 --dry-run
set -euo pipefail
# Defaults
MODE=""
TARGET=""
DRY_RUN=false
VERBOSE=false
OUTPUT=""
MODEL="${DROID_MODEL:-claude-sonnet-4-5-20250929}"
OUTPUT_MODE="" # batch or immediate, auto-detected if not set
ADD_PREVIEW_CALLOUT="" # auto-detected if not set
# Colors for output
RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[0;33m'
BLUE='\033[0;34m'
NC='\033[0m' # No Color
log() {
if [[ "$VERBOSE" == "true" ]]; then
echo -e "${BLUE}[docs-suggest]${NC} $*" >&2
fi
}
error() {
echo -e "${RED}Error:${NC} $*" >&2
exit 1
}
warn() {
echo -e "${YELLOW}Warning:${NC} $*" >&2
}
# Parse arguments
while [[ $# -gt 0 ]]; do
case $1 in
--pr)
MODE="pr"
TARGET="$2"
shift 2
;;
--commit)
MODE="commit"
TARGET="$2"
shift 2
;;
--diff)
MODE="diff"
TARGET="$2"
shift 2
;;
--staged)
MODE="staged"
shift
;;
--batch)
OUTPUT_MODE="batch"
shift
;;
--immediate)
OUTPUT_MODE="immediate"
shift
;;
--preview)
ADD_PREVIEW_CALLOUT="true"
shift
;;
--no-preview)
ADD_PREVIEW_CALLOUT="false"
shift
;;
--dry-run)
DRY_RUN=true
shift
;;
--verbose)
VERBOSE=true
shift
;;
--output)
OUTPUT="$2"
shift 2
;;
--model)
MODEL="$2"
shift 2
;;
-h|--help)
head -42 "$0" | tail -40
exit 0
;;
*)
error "Unknown option: $1"
;;
esac
done
# Validate mode
if [[ -z "$MODE" ]]; then
error "Must specify one of: --pr, --commit, --diff, or --staged"
fi
# Get repo root
REPO_ROOT="$(cd "$(dirname "$0")/.." && pwd)"
cd "$REPO_ROOT"
# Batch file location
BATCH_DIR="$REPO_ROOT/docs/.suggestions"
BATCH_FILE="$BATCH_DIR/pending.md"
# Temp directory for context assembly
TMPDIR=$(mktemp -d)
trap 'rm -rf "$TMPDIR"' EXIT
log "Mode: $MODE, Target: ${TARGET:-staged changes}"
log "Temp dir: $TMPDIR"
# ============================================================================
# Step 1: Get the diff and detect context
# ============================================================================
get_diff() {
case $MODE in
pr)
if ! command -v gh &> /dev/null; then
error "gh CLI required for --pr mode. Install: https://cli.github.com"
fi
log "Fetching PR #$TARGET info..."
# Get PR metadata for auto-detection
PR_JSON=$(gh pr view "$TARGET" --json baseRefName,title,number)
PR_BASE=$(echo "$PR_JSON" | grep -o '"baseRefName":"[^"]*"' | cut -d'"' -f4)
PR_TITLE=$(echo "$PR_JSON" | grep -o '"title":"[^"]*"' | cut -d'"' -f4)
PR_NUMBER=$(echo "$PR_JSON" | grep -o '"number":[0-9]*' | cut -d':' -f2)
log "PR #$PR_NUMBER: $PR_TITLE (base: $PR_BASE)"
# Auto-detect output mode based on target branch
if [[ -z "$OUTPUT_MODE" ]]; then
if [[ "$PR_BASE" == "main" ]]; then
OUTPUT_MODE="batch"
log "Auto-detected: batch mode (PR targets main)"
else
OUTPUT_MODE="immediate"
log "Auto-detected: immediate mode (PR targets $PR_BASE)"
fi
fi
# Auto-detect preview callout
if [[ -z "$ADD_PREVIEW_CALLOUT" ]]; then
if [[ "$PR_BASE" == "main" ]]; then
ADD_PREVIEW_CALLOUT="true"
log "Auto-detected: will add preview callout (new feature going to main)"
else
# Cherry-pick to release branch - check if it's preview or stable
ADD_PREVIEW_CALLOUT="false"
log "Auto-detected: no preview callout (cherry-pick)"
fi
fi
# Store metadata for batch mode
echo "$PR_NUMBER" > "$TMPDIR/pr_number"
echo "$PR_TITLE" > "$TMPDIR/pr_title"
echo "$PR_BASE" > "$TMPDIR/pr_base"
log "Fetching PR #$TARGET diff..."
gh pr diff "$TARGET" > "$TMPDIR/changes.diff"
gh pr diff "$TARGET" --name-only > "$TMPDIR/changed_files.txt"
;;
commit)
log "Getting commit $TARGET diff..."
git show "$TARGET" --format="" > "$TMPDIR/changes.diff"
git show "$TARGET" --format="" --name-only > "$TMPDIR/changed_files.txt"
# Default to immediate for commits
OUTPUT_MODE="${OUTPUT_MODE:-immediate}"
ADD_PREVIEW_CALLOUT="${ADD_PREVIEW_CALLOUT:-false}"
;;
diff)
if [[ ! -f "$TARGET" ]]; then
error "Diff file not found: $TARGET"
fi
log "Using provided diff file..."
cp "$TARGET" "$TMPDIR/changes.diff"
grep -E '^\+\+\+ b/' "$TARGET" | sed 's|^+++ b/||' > "$TMPDIR/changed_files.txt" || true
OUTPUT_MODE="${OUTPUT_MODE:-immediate}"
ADD_PREVIEW_CALLOUT="${ADD_PREVIEW_CALLOUT:-false}"
;;
staged)
log "Getting staged changes..."
git diff --cached > "$TMPDIR/changes.diff"
git diff --cached --name-only > "$TMPDIR/changed_files.txt"
OUTPUT_MODE="${OUTPUT_MODE:-immediate}"
ADD_PREVIEW_CALLOUT="${ADD_PREVIEW_CALLOUT:-false}"
;;
esac
if [[ ! -s "$TMPDIR/changes.diff" ]]; then
error "No changes found"
fi
log "Found $(wc -l < "$TMPDIR/changed_files.txt" | tr -d ' ') changed files"
log "Output mode: $OUTPUT_MODE, Preview callout: $ADD_PREVIEW_CALLOUT"
}
# ============================================================================
# Step 2: Filter to relevant changes
# ============================================================================
filter_changes() {
log "Filtering to documentation-relevant changes..."
# Keep only source code changes (not tests, not CI, not docs themselves)
grep -E '^crates/.*\.rs$' "$TMPDIR/changed_files.txt" | \
grep -v '_test\.rs$' | \
grep -v '/tests/' | \
grep -v '/test_' > "$TMPDIR/source_files.txt" || true
# Also track if settings/keybindings changed
grep -E '(settings|keymap|actions)' "$TMPDIR/changed_files.txt" > "$TMPDIR/config_files.txt" || true
local source_count=$(wc -l < "$TMPDIR/source_files.txt" | tr -d ' ')
local config_count=$(wc -l < "$TMPDIR/config_files.txt" | tr -d ' ')
log "Relevant files: $source_count source, $config_count config"
if [[ "$source_count" -eq 0 && "$config_count" -eq 0 ]]; then
echo "No documentation-relevant changes detected (only tests, CI, or docs modified)."
exit 0
fi
}
# ============================================================================
# Step 3: Assemble context
# ============================================================================
assemble_context() {
log "Assembling context..."
# Start the prompt
cat > "$TMPDIR/prompt.md" << 'PROMPT_HEADER'
# Documentation Suggestion Request
You are analyzing code changes to determine if documentation updates are needed.
## Your Task
1. Analyze the diff below for user-facing changes
2. Determine if any documentation updates are warranted
3. If yes, provide specific, actionable suggestions
4. If no, explain why no updates are needed
## Guidelines
PROMPT_HEADER
# Add conventions
log "Adding documentation conventions..."
cat >> "$TMPDIR/prompt.md" << 'CONVENTIONS'
### Documentation Conventions
- **Voice**: Second person ("you"), present tense, direct and concise
- **No hedging**: Avoid "simply", "just", "easily"
- **Settings pattern**: Show Settings Editor UI first, then JSON as alternative
- **Keybindings**: Use `{#kb action::Name}` syntax, not hardcoded keys
- **Terminology**: "folder" not "directory", "project" not "workspace", "Settings Editor" not "settings UI"
- **SEO keyword targeting**: For each docs page you suggest updating, choose one
primary keyword/intent phrase using the page's user intent
- **SEO metadata**: Every updated/new docs page should include frontmatter with
`title` and `description` (single-line `key: value` entries)
- **Metadata quality**: Titles should clearly state page intent (~50-60 chars),
descriptions should summarize the reader outcome (~140-160 chars)
- **Keyword usage**: Use the primary keyword naturally in frontmatter and in page
body at least once; never keyword-stuff
- **SEO structure**: Keep exactly one H1 and preserve logical H1→H2→H3
hierarchy
- **Internal links minimum**: Non-reference pages should include at least 3
useful internal docs links; reference pages can include fewer when extra links
would be noise
- **Marketing links**: For main feature pages, include a relevant `zed.dev`
marketing link alongside docs links
### Brand Voice Rubric (Required)
For suggested doc text, apply the brand rubric scoring exactly and only pass text
that scores 4+ on every criterion:
| Criterion |
| -------------------- |
| Technical Grounding |
| Natural Syntax |
| Quiet Confidence |
| Developer Respect |
| Information Priority |
| Specificity |
| Voice Consistency |
| Earned Claims |
Pass threshold: all criteria 4+ (minimum 32/40 total).
Also reject suggestions containing obvious taboo phrasing (hype, emotional
manipulation, or marketing-style superlatives).
For every docs file you suggest changing, treat the entire file as in scope for
brand review (not only the edited section). Include any additional full-file
voice fixes needed to reach passing rubric scores.
### What Requires Documentation
- New user-facing features or commands
- Changed keybindings or default behaviors
- New or modified settings
- Deprecated or removed functionality
### What Does NOT Require Documentation
- Internal refactoring without behavioral changes
- Performance optimizations (unless user-visible)
- Bug fixes that restore documented behavior
- Test changes, CI changes
CONVENTIONS
# Add preview callout instructions if needed
if [[ "$ADD_PREVIEW_CALLOUT" == "true" ]]; then
# Get current preview version for modification callouts
local preview_version
preview_version=$(gh release list --limit 5 2>/dev/null | grep -E '\-pre\s' | head -1 | grep -oE 'v[0-9]+\.[0-9]+' | head -1 || echo "v0.XXX")
preview_version="${preview_version#v}" # Remove leading 'v'
cat >> "$TMPDIR/prompt.md" << PREVIEW_INSTRUCTIONS
### Preview Release Callouts
This change is going into Zed Preview first. Use the appropriate callout based on the type of change:
#### For NEW/ADDITIVE features (new commands, new settings, new UI elements):
\`\`\`markdown
> **Preview:** This feature is available in Zed Preview. It will be included in the next Stable release.
\`\`\`
#### For BEHAVIOR MODIFICATIONS (changed defaults, altered behavior of existing features):
\`\`\`markdown
> **Changed in Preview (v${preview_version}).** See [release notes](/releases#${preview_version}).
\`\`\`
**Guidelines:**
- Use the "Preview" callout for entirely new features or sections
- Use the "Changed in Preview" callout when modifying documentation of existing behavior
- Place callouts immediately after the section heading, before any content
- Both callout types will be stripped when the feature ships to Stable
PREVIEW_INSTRUCTIONS
fi
echo "" >> "$TMPDIR/prompt.md"
echo "## Changed Files" >> "$TMPDIR/prompt.md"
echo "" >> "$TMPDIR/prompt.md"
echo '```' >> "$TMPDIR/prompt.md"
cat "$TMPDIR/changed_files.txt" >> "$TMPDIR/prompt.md"
echo '```' >> "$TMPDIR/prompt.md"
echo "" >> "$TMPDIR/prompt.md"
# Add the diff (truncated if huge)
echo "## Code Diff" >> "$TMPDIR/prompt.md"
echo "" >> "$TMPDIR/prompt.md"
local diff_lines=$(wc -l < "$TMPDIR/changes.diff" | tr -d ' ')
if [[ "$diff_lines" -gt 2000 ]]; then
warn "Diff is large ($diff_lines lines), truncating to 2000 lines"
echo '```diff' >> "$TMPDIR/prompt.md"
head -2000 "$TMPDIR/changes.diff" >> "$TMPDIR/prompt.md"
echo "" >> "$TMPDIR/prompt.md"
echo "[... truncated, $((diff_lines - 2000)) more lines ...]" >> "$TMPDIR/prompt.md"
echo '```' >> "$TMPDIR/prompt.md"
else
echo '```diff' >> "$TMPDIR/prompt.md"
cat "$TMPDIR/changes.diff" >> "$TMPDIR/prompt.md"
echo '```' >> "$TMPDIR/prompt.md"
fi
# Add output format instructions
cat >> "$TMPDIR/prompt.md" << 'OUTPUT_FORMAT'
## Output Format
Respond with ONE of these formats:
### If documentation updates ARE needed:
```markdown
## Documentation Suggestions
### Summary
[1-2 sentence summary of what changed and why docs need updating]
### Suggested Changes
#### 1. [docs/src/path/to/file.md]
- **Section**: [existing section to update, or "New section"]
- **Change**: [Add/Update/Remove]
- **Target keyword**: [single keyword/intent phrase for this page]
- **Frontmatter**:
```yaml
---
title: ...
description: ...
---
```
- **Links**: [List at least 3 internal docs links for non-reference pages; if
this is a main feature page, include one relevant `zed.dev` marketing link]
- **Suggestion**: [Specific text or description of what to add/change]
- **Full-file brand pass**: [Required: yes. Note any additional voice edits
elsewhere in the same file needed to pass rubric across the entire file.]
- **Brand voice scorecard**:
| Criterion | Score | Notes |
| -------------------- | ----- | ----- |
| Technical Grounding | /5 | |
| Natural Syntax | /5 | |
| Quiet Confidence | /5 | |
| Developer Respect | /5 | |
| Information Priority | /5 | |
| Specificity | /5 | |
| Voice Consistency | /5 | |
| Earned Claims | /5 | |
| **TOTAL** | /40 | |
Pass threshold: all criteria 4+.
#### 2. [docs/src/another/file.md]
...
### Notes for Reviewer
[Any context or uncertainty worth flagging]
```
### If NO documentation updates are needed:
```markdown
## No Documentation Updates Needed
**Reason**: [Brief explanation - e.g., "Internal refactoring only", "Test changes", "Bug fix restoring existing behavior"]
**Changes reviewed**:
- [Brief summary of what the code changes do]
- [Why they don't affect user-facing documentation]
```
Be conservative. Only suggest documentation changes when there's a clear user-facing impact.
OUTPUT_FORMAT
log "Context assembled: $(wc -l < "$TMPDIR/prompt.md" | tr -d ' ') lines"
}
# ============================================================================
# Step 4: Run the analysis
# ============================================================================
run_analysis() {
if [[ "$DRY_RUN" == "true" ]]; then
echo -e "${GREEN}=== DRY RUN: Assembled Context ===${NC}"
echo ""
echo "Output mode: $OUTPUT_MODE"
echo "Preview callout: $ADD_PREVIEW_CALLOUT"
if [[ "$OUTPUT_MODE" == "batch" ]]; then
echo "Batch file: $BATCH_FILE"
fi
echo ""
cat "$TMPDIR/prompt.md"
echo ""
echo -e "${GREEN}=== End Context ===${NC}"
echo ""
echo "To run for real, remove --dry-run flag"
return
fi
# Check for droid CLI
if ! command -v droid &> /dev/null; then
error "droid CLI required. Install from: https://app.factory.ai/cli"
fi
log "Running analysis with model: $MODEL"
# Run the LLM
local suggestions
suggestions=$(droid exec -m "$MODEL" -f "$TMPDIR/prompt.md")
# Handle output based on mode
if [[ "$OUTPUT_MODE" == "batch" ]]; then
append_to_batch "$suggestions"
else
output_immediate "$suggestions"
fi
}
# ============================================================================
# Output handlers
# ============================================================================
append_to_batch() {
local suggestions="$1"
# Check if suggestions indicate no updates needed
if echo "$suggestions" | grep -q "No Documentation Updates Needed"; then
log "No documentation updates needed, skipping batch"
echo "$suggestions"
return
fi
# Create batch directory if needed
mkdir -p "$BATCH_DIR"
# Get PR info if available
local pr_number=""
local pr_title=""
if [[ -f "$TMPDIR/pr_number" ]]; then
pr_number=$(cat "$TMPDIR/pr_number")
pr_title=$(cat "$TMPDIR/pr_title")
fi
# Initialize batch file if it doesn't exist
if [[ ! -f "$BATCH_FILE" ]]; then
cat > "$BATCH_FILE" << 'BATCH_HEADER'
# Pending Documentation Suggestions
This file contains batched documentation suggestions for the next Preview release.
Run `script/docs-suggest-publish` to create a PR from these suggestions.
---
BATCH_HEADER
fi
# Append suggestions with metadata
{
echo ""
echo "## PR #$pr_number: $pr_title"
echo ""
echo "_Added: $(date -u +%Y-%m-%dT%H:%M:%SZ)_"
echo ""
echo "$suggestions"
echo ""
echo "---"
} >> "$BATCH_FILE"
echo -e "${GREEN}Suggestions batched to:${NC} $BATCH_FILE"
echo ""
echo "Batched suggestions for PR #$pr_number"
echo "Run 'script/docs-suggest-publish' when ready to create the docs PR."
}
output_immediate() {
local suggestions="$1"
if [[ -n "$OUTPUT" ]]; then
echo "$suggestions" > "$OUTPUT"
echo "Suggestions written to: $OUTPUT"
else
echo "$suggestions"
fi
}
# ============================================================================
# Main
# ============================================================================
main() {
get_diff
filter_changes
assemble_context
run_analysis
}
main

378
script/docs-suggest-publish Executable file
View file

@ -0,0 +1,378 @@
#!/usr/bin/env bash
#
# Create a draft documentation PR by auto-applying batched suggestions.
#
# Usage:
# script/docs-suggest-publish [--dry-run] [--model MODEL]
#
# This script:
# 1. Reads pending suggestions from the docs/suggestions-pending branch
# 2. Uses Droid to apply all suggestions directly to docs files
# 3. Runs docs formatting
# 4. Creates a draft PR for human review/merge
# 5. Optionally resets the suggestions branch after successful PR creation
#
# Options:
# --dry-run Show what would be done without creating PR
# --keep-queue Don't reset the suggestions branch after PR creation
# --model MODEL Override Droid model used for auto-apply
# --verbose Show detailed progress
#
# Run this as part of the preview release workflow.
set -euo pipefail
DRY_RUN=false
KEEP_QUEUE=false
VERBOSE=false
MODEL="${DROID_MODEL:-claude-sonnet-4-5-20250929}"
SUGGESTIONS_BRANCH="docs/suggestions-pending"
# Colors
RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[0;33m'
BLUE='\033[0;34m'
NC='\033[0m'
log() {
if [[ "$VERBOSE" == "true" ]]; then
echo -e "${BLUE}[docs-publish]${NC} $*" >&2
fi
}
error() {
echo -e "${RED}Error:${NC} $*" >&2
exit 1
}
# Parse arguments
while [[ $# -gt 0 ]]; do
case $1 in
--dry-run)
DRY_RUN=true
shift
;;
--keep-queue)
KEEP_QUEUE=true
shift
;;
--verbose)
VERBOSE=true
shift
;;
--model)
MODEL="$2"
shift 2
;;
-h|--help)
head -26 "$0" | tail -24
exit 0
;;
*)
error "Unknown option: $1"
;;
esac
done
# Get repo root
REPO_ROOT="$(cd "$(dirname "$0")/.." && pwd)"
cd "$REPO_ROOT"
# Check if suggestions branch exists
log "Checking for suggestions branch..."
if ! git ls-remote --exit-code --heads origin "$SUGGESTIONS_BRANCH" > /dev/null 2>&1; then
echo "No pending suggestions found (branch $SUGGESTIONS_BRANCH doesn't exist)."
echo "Suggestions are queued automatically when PRs are merged to main."
exit 0
fi
# Fetch the suggestions branch
log "Fetching suggestions branch..."
git fetch origin "$SUGGESTIONS_BRANCH"
# Check for manifest
if ! git show "origin/$SUGGESTIONS_BRANCH:manifest.json" > /dev/null 2>&1; then
echo "No manifest found on suggestions branch."
exit 0
fi
# Read manifest
MANIFEST=$(git show "origin/$SUGGESTIONS_BRANCH:manifest.json")
SUGGESTION_COUNT=$(echo "$MANIFEST" | jq '.suggestions | length')
if [[ "$SUGGESTION_COUNT" -eq 0 ]]; then
echo "No pending suggestions in queue."
exit 0
fi
echo "Found $SUGGESTION_COUNT pending suggestion(s):"
echo ""
echo "$MANIFEST" | jq -r '.suggestions[] | " PR #\(.pr): \(.title)"'
echo ""
if [[ "$DRY_RUN" == "true" ]]; then
echo -e "${YELLOW}=== DRY RUN ===${NC}"
echo ""
echo "Would auto-apply suggestions to docs via Droid and create a draft PR."
echo "Model: $MODEL"
echo ""
# Show each suggestion file
for file in $(echo "$MANIFEST" | jq -r '.suggestions[].file'); do
echo "--- $file ---"
git show "origin/$SUGGESTIONS_BRANCH:$file" 2>/dev/null || echo "(file not found)"
echo ""
done
echo -e "${YELLOW}=== END DRY RUN ===${NC}"
echo ""
echo "Run without --dry-run to create the PR."
exit 0
fi
# Ensure clean working state
if [[ -n "$(git status --porcelain)" ]]; then
error "Working directory has uncommitted changes. Please commit or stash first."
fi
for command in git gh jq droid; do
if ! command -v "$command" > /dev/null 2>&1; then
error "Required command not found: $command"
fi
done
# Remember current branch
ORIGINAL_BRANCH=$(git branch --show-current)
log "Current branch: $ORIGINAL_BRANCH"
# Create new branch for docs PR from latest main
git fetch origin main
DOCS_BRANCH="docs/preview-auto-$(date +%Y-%m-%d-%H%M%S)"
log "Creating docs branch: $DOCS_BRANCH"
git checkout -b "$DOCS_BRANCH" origin/main
TMPDIR=$(mktemp -d)
trap 'rm -rf "$TMPDIR"' EXIT
SUGGESTIONS_FILE="$TMPDIR/suggestions.md"
APPLY_PROMPT_FILE="$TMPDIR/apply-prompt.md"
APPLY_SUMMARY_FILE="$TMPDIR/apply-summary.md"
# Combine queued suggestion files into one input
for file in $(echo "$MANIFEST" | jq -r '.suggestions[].file'); do
{
echo "## Source: $file"
echo ""
git show "origin/$SUGGESTIONS_BRANCH:$file" 2>/dev/null || error "Suggestion file missing: $file"
echo ""
echo "---"
echo ""
} >> "$SUGGESTIONS_FILE"
done
# Build auto-apply prompt
cat > "$APPLY_PROMPT_FILE" << 'EOF'
# Documentation Auto-Apply Request (Preview Release)
Apply all queued documentation suggestions below directly to docs files in this repository.
## Required behavior
1. Apply concrete documentation edits (not suggestion text) to the appropriate files.
2. Edit only docs content files under `docs/src/` unless a suggestion explicitly requires another docs path.
3. For every docs file you modify, run a full-file brand voice pass (entire file, not only edited sections).
4. Enforce the brand rubric exactly; final file content must score 4+ on every criterion:
- Technical Grounding
- Natural Syntax
- Quiet Confidence
- Developer Respect
- Information Priority
- Specificity
- Voice Consistency
- Earned Claims
5. Keep SEO/frontmatter/linking requirements from the suggestions where applicable.
6. Keep preview callout semantics correct:
- Additive features: `> **Preview:** ...`
- Behavior modifications: `> **Changed in Preview (vX.XXX).** ...`
7. If a suggestion is too ambiguous to apply safely, skip it and explain why in the summary.
## Output format (after making edits)
Return markdown with:
- `## Applied Suggestions`
- `## Skipped Suggestions`
- `## Files Updated`
- `## Brand Voice Verification` (one line per updated file confirming full-file pass)
Do not include a patch in the response; apply edits directly to files.
## Queued Suggestions
EOF
cat "$SUGGESTIONS_FILE" >> "$APPLY_PROMPT_FILE"
log "Running Droid auto-apply with model: $MODEL"
droid exec -m "$MODEL" -f "$APPLY_PROMPT_FILE" > "$APPLY_SUMMARY_FILE"
if [[ -n "$(git status --porcelain | grep -vE '^.. docs/' || true)" ]]; then
error "Auto-apply modified non-doc files. Revert and re-run."
fi
if [[ -z "$(git status --porcelain docs/ | grep '^.. docs/src/' || true)" ]]; then
error "Auto-apply produced no docs/src changes."
fi
log "Running docs formatter"
./script/prettier
if [[ -z "$(git status --porcelain docs/ | grep '^.. docs/src/' || true)" ]]; then
error "No docs/src changes remain after formatting; aborting PR creation."
fi
# Build PR body from suggestions
PR_BODY_FILE="$TMPDIR/pr-body.md"
cat > "$PR_BODY_FILE" << 'EOF'
# Documentation Updates for Preview Release
This draft PR auto-applies queued documentation suggestions collected from
recently merged PRs.
## How to Use This PR
1. Review the applied changes file-by-file.
2. Verify brand voice on each touched file as a full-file pass.
3. Ensure docs use the correct callout type:
- Additive features: Preview callout
- Behavior modifications: Changed in Preview callout
4. Merge when ready.
## Auto-Apply Summary
EOF
cat "$APPLY_SUMMARY_FILE" >> "$PR_BODY_FILE"
cat >> "$PR_BODY_FILE" << 'EOF'
## Preview Callouts
Use the Preview callout for new/additive features:
```markdown
> **Preview:** This feature is available in Zed Preview. It will be included in the next Stable release.
```
Use this callout for behavior modifications to existing functionality:
```markdown
> **Changed in Preview (v0.XXX).** See [release notes](/releases#0.XXX).
```
---
## Pending Suggestions
EOF
# Append each suggestion to PR body
for file in $(echo "$MANIFEST" | jq -r '.suggestions[].file'); do
log "Adding $file to PR body..."
echo "" >> "$PR_BODY_FILE"
git show "origin/$SUGGESTIONS_BRANCH:$file" >> "$PR_BODY_FILE" 2>/dev/null || true
echo "" >> "$PR_BODY_FILE"
echo "---" >> "$PR_BODY_FILE"
done
# Add tracking info
cat >> "$PR_BODY_FILE" << EOF
## PRs Included
EOF
echo "$MANIFEST" | jq -r '.suggestions[] | "- [PR #\(.pr)](\(.file)): \(.title)"' >> "$PR_BODY_FILE"
git add docs/
git commit -m "docs: auto-apply preview release suggestions
Auto-applied queued documentation suggestions from:
$(echo "$MANIFEST" | jq -r '.suggestions[] | "- PR #\(.pr)"')
Generated with script/docs-suggest-publish for human review in draft PR."
# Push and create PR
log "Pushing branch..."
git push -u origin "$DOCS_BRANCH"
log "Creating PR..."
PR_URL=$(gh pr create \
--draft \
--title "docs: auto-apply preview release suggestions" \
--body-file "$PR_BODY_FILE" \
--label "documentation")
echo ""
echo -e "${GREEN}PR created:${NC} $PR_URL"
# Reset suggestions branch if not keeping
if [[ "$KEEP_QUEUE" != "true" ]]; then
echo ""
echo "Resetting suggestions queue..."
git checkout --orphan "${SUGGESTIONS_BRANCH}-reset"
git rm -rf . > /dev/null 2>&1 || true
cat > README.md << 'EOF'
# Documentation Suggestions Queue
This branch contains batched documentation suggestions for the next Preview release.
Each file represents suggestions from a merged PR. At preview branch cut time,
run `script/docs-suggest-publish` to create a documentation PR from these suggestions.
## Structure
- `suggestions/PR-XXXXX.md` - Suggestions for PR #XXXXX
- `manifest.json` - Index of all pending suggestions
## Workflow
1. PRs merged to main trigger documentation analysis
2. Suggestions are committed here as individual files
3. At preview release, suggestions are collected into a docs PR
4. After docs PR is created, this branch is reset
EOF
mkdir -p suggestions
echo '{"suggestions":[]}' > manifest.json
git add README.md suggestions manifest.json
git commit -m "Reset documentation suggestions queue
Previous suggestions published in: $PR_URL"
git push -f origin "${SUGGESTIONS_BRANCH}-reset:$SUGGESTIONS_BRANCH"
git checkout "$ORIGINAL_BRANCH"
git branch -D "${SUGGESTIONS_BRANCH}-reset"
echo "Suggestions queue reset."
else
git checkout "$ORIGINAL_BRANCH"
echo ""
echo "Suggestions queue kept (--keep-queue). Remember to reset manually after PR is merged."
fi
# Cleanup
echo ""
echo -e "${GREEN}Done!${NC}"
echo ""
echo "Next steps:"
echo "1. Review the draft PR and verify all auto-applied docs changes"
echo "2. Confirm full-file brand voice pass for each touched docs file"
echo "3. Mark ready for review and merge"

139
script/test-docs-suggest-batch Executable file
View file

@ -0,0 +1,139 @@
#!/usr/bin/env bash
#
# Test docs-suggest on multiple PRs and generate a summary report.
#
# Usage:
# script/test-docs-suggest-batch [--limit N] [--output FILE]
#
# This script runs docs-suggest in dry-run mode on recent merged PRs
# to validate the context assembly and help tune the prompt.
set -euo pipefail
LIMIT=50
OUTPUT="docs-suggest-batch-results.md"
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
while [[ $# -gt 0 ]]; do
case $1 in
--limit)
LIMIT="$2"
shift 2
;;
--output)
OUTPUT="$2"
shift 2
;;
*)
echo "Unknown option: $1"
exit 1
;;
esac
done
echo "Testing docs-suggest on $LIMIT recent merged PRs..."
echo "Output: $OUTPUT"
echo ""
# Get list of PRs
PRS=$(gh pr list --state merged --limit "$LIMIT" --json number,title --jq '.[] | "\(.number)|\(.title)"')
# Initialize output file
cat > "$OUTPUT" << HEADER
# docs-suggest Batch Test Results
**Date**: $(date +%Y-%m-%d)
**PRs tested**: $LIMIT
## Summary
| PR | Title | Result | Source Files | Notes |
|----|-------|--------|--------------|-------|
HEADER
# Track stats
total=0
has_source=0
no_source=0
errors=0
while IFS='|' read -r pr_num title; do
total=$((total + 1))
echo -n "[$total/$LIMIT] PR #$pr_num: "
# Run dry-run and capture output
tmpfile=$(mktemp)
if "$SCRIPT_DIR/docs-suggest" --pr "$pr_num" --dry-run 2>"$tmpfile.err" >"$tmpfile.out"; then
# Check if it found source files
if grep -q "No documentation-relevant changes" "$tmpfile.out"; then
result="No source changes"
no_source=$((no_source + 1))
source_count="0"
echo "skipped (no source)"
else
# Extract source file count from verbose output
source_count=$(grep -oE '[0-9]+ source' "$tmpfile.err" 2>/dev/null | grep -oE '[0-9]+' || echo "?")
result="Has source changes"
has_source=$((has_source + 1))
echo "has $source_count source files"
fi
notes=""
else
result="Error"
errors=$((errors + 1))
source_count="-"
notes=$(head -1 "$tmpfile.err" 2>/dev/null || echo "unknown error")
echo "error"
fi
# Escape title for markdown table
title_escaped=$(echo "$title" | sed 's/|/\\|/g' | cut -c1-60)
# Add row to table
echo "| [#$pr_num](https://github.com/zed-industries/zed/pull/$pr_num) | $title_escaped | $result | $source_count | $notes |" >> "$OUTPUT"
rm -f "$tmpfile" "$tmpfile.out" "$tmpfile.err"
done <<< "$PRS"
# Add summary stats
cat >> "$OUTPUT" << STATS
## Statistics
- **Total PRs**: $total
- **With source changes**: $has_source ($(( has_source * 100 / total ))%)
- **No source changes**: $no_source ($(( no_source * 100 / total ))%)
- **Errors**: $errors
## Observations
_Add manual observations here after reviewing results._
## Sample Contexts
STATS
# Add 3 sample contexts from PRs with source changes
echo "" >> "$OUTPUT"
echo "### Sample 1: PR with source changes" >> "$OUTPUT"
echo "" >> "$OUTPUT"
sample_pr=$(gh pr list --state merged --limit 20 --json number --jq '.[].number' | while read pr; do
if "$SCRIPT_DIR/docs-suggest" --pr "$pr" --dry-run 2>/dev/null | grep -q "## Code Diff"; then
echo "$pr"
break
fi
done)
if [[ -n "$sample_pr" ]]; then
echo "PR #$sample_pr:" >> "$OUTPUT"
echo "" >> "$OUTPUT"
echo '```' >> "$OUTPUT"
"$SCRIPT_DIR/docs-suggest" --pr "$sample_pr" --dry-run 2>/dev/null | head -100 >> "$OUTPUT"
echo '```' >> "$OUTPUT"
fi
echo ""
echo "Done! Results written to: $OUTPUT"
echo ""
echo "Stats: $has_source with source changes, $no_source without, $errors errors"