From 7312c645801e24093c24b6a469eff4a4164a4ffe Mon Sep 17 00:00:00 2001 From: lefarcen <935902669@qq.com> Date: Tue, 26 May 2026 22:05:04 +0800 Subject: [PATCH] ci(landing): split landing deploy into staging gate + manual production (#2994) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * ci(landing): split landing deploy into staging gate + manual production A merge to `main` previously published the landing page straight to production (open-design.ai) via `landing-page-deploy`. There was no buffer to review the rendered site, so a bad merge was live instantly. Split deploys across two Cloudflare Pages projects so production is only ever reached by an explicit human action: - `landing-page-staging` (push to main) -> staging project `open-design-landing-staging` -> staging.open-design.ai. - `landing-page-production` (manual workflow_dispatch only) -> production project `open-design-landing` -> open-design.ai. Only this workflow names the production project; gate it with required reviewers on the `production` GitHub environment. - `landing-page-ci` now also deploys a per-PR preview into the staging project (`--branch=pr-`) for same-repo branches and comments the URL. Fork PRs (no secrets / read-only token) skip the deploy and keep just the build validation. Path filters already scope this to landing edits. Decouple search-engine indexing from staging: - `blog-indexing-on-deploy` now triggers on `landing-page-production` (not every main push), so the test environment is never submitted to Google/IndexNow. - It diffs from a new `blog-indexed-prod` tag (the last indexed prod commit) instead of `HEAD^`, and force-advances the tag after a successful run, so a manual promotion bundling several merged posts indexes all of them rather than only the last commit. Staging and PR-preview builds drop `PUBLIC_GA_MEASUREMENT_ID` so test traffic does not pollute the production GA property. * ci(landing): keep staging + PR previews out of the search index staging.open-design.ai mirrors production and is exposed via cert transparency logs, so search engines can discover it. Indexing the mirror competes with open-design.ai for the same content. Emit `` whenever OD_LANDING_NOINDEX=1, and set that flag on the staging and PR-preview builds (production leaves it unset and stays indexable). noindex is used rather than a robots.txt Disallow so crawlers can still fetch the page and read both the tag and the canonical, which already points at the production origin. * fix(landing): make staging noindex actually take effect The previous commit read `process.env.OD_LANDING_NOINDEX` directly in `seo-head.astro`, but `.astro` frontmatter is transformed by Vite and does not see process.env, so the meta never rendered. Two fixes: - Inject the flag as the compile-time constant `__OD_LANDING_NOINDEX__` via `vite.define` in astro.config.ts (config runs in Node and can read process.env); SeoHead consumes that constant. - The homepage (`index.astro`) and `og.astro` build their own and never use SeoHead, so a per-component meta can miss pages. Add an `astro:build:done` integration that appends a catch-all `/* X-Robots-Tag: noindex, nofollow` to the Cloudflare Pages `_headers` on staging/preview builds, covering every response (homepage, assets, any custom-head page) at the HTTP layer. Production builds leave `_headers` untouched. Verified: build with OD_LANDING_NOINDEX=1 emits the _headers block and the SeoHead ; build without the flag emits neither; astro check clean. * fix(landing): address review β€” pin prod checkout to main, defer index pointer Two blockers from review: - landing-page-production: workflow_dispatch can be launched from any ref via the Actions "Use workflow from" dropdown, so an operator could ship an arbitrary branch to open-design.ai. Pin the checkout to `ref: main` so the deployed artifact always equals reviewed main. - blog-indexing-on-deploy: the `blog-indexed-prod` pointer was advanced right after sitemap submission, before Inspect / Search Analytics / Render status / Open status PR. A failure in any of those still moved the pointer, so the next production run skipped those posts. Move the advance to the very end, gated on `success()`, so a failure leaves the tag in place and the range is re-processed next run (submissions are idempotent). * fix(landing): gate production promotion to the main ref only Follow-up to the production-path review note: pinning checkout to main fixed the deployed content, but the workflow was still dispatchable from any ref, which records a non-main production run and would dodge blog-indexing's `workflow_run` `branches: [main]` filter. Gate the whole job on `github.ref == 'refs/heads/main'` so a dispatch from any other branch/tag is skipped outright. --- .github/workflows/blog-indexing-on-deploy.yml | 43 ++++- .github/workflows/ci.yml | 2 +- .github/workflows/landing-page-ci.yml | 87 +++++++++- .github/workflows/landing-page-production.yml | 156 ++++++++++++++++++ ...ge-deploy.yml => landing-page-staging.yml} | 37 ++++- apps/landing-page/AGENTS.md | 36 +++- .../app/_components/seo-head.astro | 13 ++ apps/landing-page/app/env.d.ts | 4 + apps/landing-page/astro.config.ts | 37 ++++- .../scripts/blog-indexing/verify-readiness.ts | 2 +- docs/blog-indexing-automation.md | 17 +- 11 files changed, 404 insertions(+), 30 deletions(-) create mode 100644 .github/workflows/landing-page-production.yml rename .github/workflows/{landing-page-deploy.yml => landing-page-staging.yml} (75%) diff --git a/.github/workflows/blog-indexing-on-deploy.yml b/.github/workflows/blog-indexing-on-deploy.yml index 215c2af3c..3043fade1 100644 --- a/.github/workflows/blog-indexing-on-deploy.yml +++ b/.github/workflows/blog-indexing-on-deploy.yml @@ -1,6 +1,8 @@ name: blog-indexing-on-deploy -# Runs after every successful `landing-page-deploy` on main. The job is +# Runs after every successful `landing-page-production` promotion. Staging +# deploys (`landing-page-staging`) intentionally do NOT trigger indexing, so +# the test environment is never submitted to search engines. The job is # idempotent and follows the blog-indexing-automation skill: # # 1. Detect blog URLs added/modified in the deploy @@ -16,7 +18,7 @@ name: blog-indexing-on-deploy on: workflow_run: - workflows: ['landing-page-deploy'] + workflows: ['landing-page-production'] types: [completed] branches: [main] workflow_dispatch: @@ -29,7 +31,10 @@ on: required: false permissions: - contents: read + # `contents: write` lets the job advance the `blog-indexed-prod` tag after a + # successful run so the next promotion diffs from exactly where this one + # stopped. The status PR uses a separate GitHub App token (below). + contents: write concurrency: group: blog-indexing-on-deploy @@ -147,8 +152,21 @@ jobs: mkdir -p .blog-indexing BASE="${{ github.event.inputs.base_sha || '' }}" if [ -z "$BASE" ]; then - BASE="$(git rev-parse HEAD^)" + # Diff from the last production deploy we already indexed, tracked + # by the `blog-indexed-prod` tag. Production is a manual promotion + # that may bundle several merged posts, so a HEAD^ diff would miss + # all but the last commit. The tag is advanced at the end of a + # successful run (see "Advance indexed-production pointer"). + git fetch --no-tags origin "+refs/tags/blog-indexed-prod:refs/tags/blog-indexed-prod" 2>/dev/null || true + BASE="$(git rev-parse --verify --quiet 'refs/tags/blog-indexed-prod^{commit}' || true)" + if [ -n "$BASE" ]; then + echo "Diffing from last indexed production commit (blog-indexed-prod): $BASE" + else + BASE="$(git rev-parse HEAD^)" + echo "No blog-indexed-prod tag yet; bootstrapping from HEAD^: $BASE" + fi fi + echo "base=$BASE" >> "$GITHUB_OUTPUT" pnpm --filter @open-design/landing-page exec tsx scripts/blog-indexing/detect-changed-urls.ts \ --base "$BASE" \ --head HEAD \ @@ -264,3 +282,20 @@ jobs: Generated by `.github/workflows/blog-indexing-on-deploy.yml`. The sidecar `docs/blog-indexing-status.json` is the canonical state; the markdown file is rendered from it. + + # Advance the pointer LAST, only after every post-deploy step above + # (detect, verify, submit, inspect, analytics, render, status PR) + # succeeded. `success()` is false if any earlier step failed, so a + # failure leaves the tag where it was and the next production run + # re-processes the same range rather than silently skipping posts. + # Skipped steps (count == 0, or GSC/bot not configured) do not count as + # failures, so the pointer still advances over an empty/partial range. + # Restricted to the real production-deploy trigger; an ad-hoc manual + # dispatch must not move the production baseline. + - name: Advance indexed-production pointer + if: success() && github.event_name == 'workflow_run' + run: | + HEAD_SHA="$(git rev-parse HEAD)" + git tag -f blog-indexed-prod "$HEAD_SHA" + git push -f origin "refs/tags/blog-indexed-prod" + echo "Advanced blog-indexed-prod -> $HEAD_SHA" diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index ef3e4ca91..c56ed31f7 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -87,7 +87,7 @@ jobs: case "$file" in *.md|*.mdx|*.txt|LICENSE|.gitignore|.editorconfig|.vscode/*|.idea/*|docs/*|.github/ISSUE_TEMPLATE/*|.github/CODEOWNERS) ;; - apps/landing-page/*|flake.nix|flake.lock|nix/*|.github/workflows/nix-check.yml|.github/workflows/landing-page-ci.yml|.github/workflows/landing-page-deploy.yml|.github/workflows/blog-indexing-on-deploy.yml|.github/workflows/blog-indexing-monitor.yml|.github/workflows/blog-3day-report.yml|.github/workflows/seo-daily-report.yml|.github/workflows/actionlint.yml|.github/workflows/visual-pr-capture.yml|.github/workflows/visual-pr-comment.yml) + apps/landing-page/*|flake.nix|flake.lock|nix/*|.github/workflows/nix-check.yml|.github/workflows/landing-page-ci.yml|.github/workflows/landing-page-staging.yml|.github/workflows/landing-page-production.yml|.github/workflows/blog-indexing-on-deploy.yml|.github/workflows/blog-indexing-monitor.yml|.github/workflows/blog-3day-report.yml|.github/workflows/seo-daily-report.yml|.github/workflows/actionlint.yml|.github/workflows/visual-pr-capture.yml|.github/workflows/visual-pr-comment.yml) ;; *) workspace_validation_required=true diff --git a/.github/workflows/landing-page-ci.yml b/.github/workflows/landing-page-ci.yml index aa688b031..17910f7c3 100644 --- a/.github/workflows/landing-page-ci.yml +++ b/.github/workflows/landing-page-ci.yml @@ -5,7 +5,8 @@ on: paths: # Workflow files - .github/workflows/landing-page-ci.yml - - .github/workflows/landing-page-deploy.yml + - .github/workflows/landing-page-staging.yml + - .github/workflows/landing-page-production.yml - .github/workflows/blog-indexing-on-deploy.yml - .github/workflows/blog-indexing-monitor.yml # Landing page sources @@ -28,7 +29,8 @@ on: - main paths: - .github/workflows/landing-page-ci.yml - - .github/workflows/landing-page-deploy.yml + - .github/workflows/landing-page-staging.yml + - .github/workflows/landing-page-production.yml - .github/workflows/blog-indexing-on-deploy.yml - .github/workflows/blog-indexing-monitor.yml - apps/landing-page/** @@ -44,6 +46,10 @@ on: permissions: contents: read + # Needed to post/update the preview-URL comment on the PR. Fork PRs run + # with a read-only token regardless, so the preview steps below are gated + # to same-repo branches. + pull-requests: write concurrency: group: landing-page-ci-${{ github.event.pull_request.number || github.ref }} @@ -94,9 +100,13 @@ jobs: - name: Generate skill + template previews run: pnpm --filter @open-design/landing-page previews + # No PUBLIC_GA_MEASUREMENT_ID for PR/CI builds: the per-PR preview must + # not report into the production GA property. OD_LANDING_NOINDEX=1 keeps + # the PR preview (pr-.open-design-landing-staging.pages.dev) out of + # search engines. Only `landing-page-production` builds without these. - name: Build landing page env: - PUBLIC_GA_MEASUREMENT_ID: ${{ vars.PUBLIC_GA_MEASUREMENT_ID }} + OD_LANDING_NOINDEX: '1' run: pnpm --filter @open-design/landing-page build:static - name: Lint changed blog SEO @@ -153,3 +163,74 @@ jobs: process.exit(1); } NODE + + # --- PR preview deploy ------------------------------------------------- + # Publish this PR's built site to its own preview URL in the STAGING + # project (`--branch=pr-`) so reviewers see the rendered result + # before merge. It lands in the staging project, never the production + # project. Gated to same-repo branches: fork PRs run without the + # Cloudflare secrets and with a read-only token, so they skip the + # deploy/comment and keep just the validation above. + - name: Deploy PR preview to Cloudflare Pages + id: preview + if: github.event_name == 'pull_request' && github.event.pull_request.head.repo.full_name == github.repository + uses: cloudflare/wrangler-action@v3 + with: + apiToken: ${{ secrets.CLOUDFLARE_API_TOKEN }} + accountId: ${{ secrets.CLOUDFLARE_ACCOUNT_ID }} + workingDirectory: apps/landing-page + packageManager: npm + command: > + pages deploy out + --project-name=open-design-landing-staging + --branch=pr-${{ github.event.pull_request.number }} + + - name: Comment preview URL on PR + if: github.event_name == 'pull_request' && github.event.pull_request.head.repo.full_name == github.repository + uses: actions/github-script@v7 + env: + DEPLOY_URL: ${{ steps.preview.outputs.deployment-url }} + ALIAS_URL: ${{ steps.preview.outputs.pages-deployment-alias-url }} + with: + script: | + const marker = ''; + const deploy = process.env.DEPLOY_URL || ''; + const alias = + process.env.ALIAS_URL || + `https://pr-${context.issue.number}.open-design-landing-staging.pages.dev`; + const sha = context.payload.pull_request.head.sha.slice(0, 7); + const body = [ + marker, + '### πŸš€ Landing page preview', + '', + 'This PR is deployed to a Cloudflare Pages preview β€” **not** staging or production:', + '', + `- Stable alias: ${alias}`, + deploy ? `- This build: ${deploy}` : '', + '', + `Updated for commit \`${sha}\`.`, + ] + .filter(Boolean) + .join('\n'); + const comments = await github.paginate(github.rest.issues.listComments, { + owner: context.repo.owner, + repo: context.repo.repo, + issue_number: context.issue.number, + per_page: 100, + }); + const existing = comments.find((c) => c.body && c.body.includes(marker)); + if (existing) { + await github.rest.issues.updateComment({ + owner: context.repo.owner, + repo: context.repo.repo, + comment_id: existing.id, + body, + }); + } else { + await github.rest.issues.createComment({ + owner: context.repo.owner, + repo: context.repo.repo, + issue_number: context.issue.number, + body, + }); + } diff --git a/.github/workflows/landing-page-production.yml b/.github/workflows/landing-page-production.yml new file mode 100644 index 000000000..f76f2cee0 --- /dev/null +++ b/.github/workflows/landing-page-production.yml @@ -0,0 +1,156 @@ +name: landing-page-production + +# Promotes the current landing page to PRODUCTION: the `open-design-landing` +# Cloudflare Pages project, served at open-design.ai. This is the ONLY +# workflow that names the production project, and it is manual-only +# (workflow_dispatch) β€” a merge to `main` can never reach production on its +# own; it only updates the staging project (staging.open-design.ai) via +# `landing-page-staging`. Gate this further by configuring required reviewers +# on the GitHub `production` environment (Settings β†’ Environments). +# +# The build is identical to staging/CI, so what you reviewed on +# staging.open-design.ai is what ships. + +on: + workflow_dispatch: + inputs: + reason: + description: 'Why promote now? (recorded in the run log)' + required: false + +permissions: + contents: read + deployments: write + +# Never cancel an in-flight production deploy. +concurrency: + group: landing-page-production + cancel-in-progress: false + +jobs: + deploy: + name: Deploy landing page to production + # Production ships `main` only. workflow_dispatch can be launched from any + # ref via the Actions "Use workflow from" dropdown; gate the whole job on + # the main ref so a dispatch from a feature branch/tag is skipped outright + # (no deploy) instead of recording a non-main production run β€” which would + # also dodge blog-indexing's `workflow_run` `branches: [main]` filter. + if: github.repository == 'nexu-io/open-design' && github.ref == 'refs/heads/main' + runs-on: ubuntu-latest + timeout-minutes: 20 + environment: + name: production + url: https://open-design.ai + + steps: + - name: Checkout + uses: actions/checkout@v6.0.2 + with: + # Production always ships `main`. workflow_dispatch can be launched + # from any ref via the Actions "Use workflow from" dropdown, so pin + # the checkout to main β€” the deployed artifact must equal reviewed + # main, never whatever branch/tag the operator happened to select. + ref: main + + - name: Setup pnpm + uses: pnpm/action-setup@v5 + with: + version: 10.33.2 + + - name: Setup Node.js + uses: actions/setup-node@v6 + with: + node-version: 24 + cache: pnpm + + - name: Install dependencies + run: pnpm install --frozen-lockfile + + - name: Resolve Playwright version + id: playwright-version + run: | + version=$(node -p "require('./apps/landing-page/package.json').devDependencies.playwright.replace(/[^0-9.]/g,'')") + echo "version=$version" >> "$GITHUB_OUTPUT" + + - name: Cache generated previews + id: previews-cache + uses: actions/cache@v5.0.5 + with: + path: apps/landing-page/public/previews + key: landing-page-previews-${{ runner.os }}-${{ hashFiles('pnpm-lock.yaml', 'package.json', 'apps/landing-page/package.json', 'apps/landing-page/scripts/generate-previews.ts', 'skills/**', 'design-templates/**', 'templates/live-artifacts/**') }} + restore-keys: | + landing-page-previews-${{ runner.os }}- + + - name: Cache Playwright browsers + uses: actions/cache@v5.0.5 + with: + path: ~/.cache/ms-playwright + key: playwright-${{ runner.os }}-${{ steps.playwright-version.outputs.version }} + + - name: Install Playwright Chromium + run: pnpm --filter @open-design/landing-page exec playwright install --with-deps chromium + + - name: Typecheck landing page + run: pnpm --filter @open-design/landing-page typecheck + + # Generate previews before build so they end up in `out/previews/`. + # Soft vs. hard failure is enforced inside the script itself: + # individual broken `example.html` entries are logged and skipped, + # but a systemic failure (chromium launch error, every job failing) + # exits non-zero so we don't silently ship a deploy with zero + # thumbnails to production. + - name: Generate skill + template previews + run: pnpm --filter @open-design/landing-page previews + + - name: Build landing page + env: + PUBLIC_GA_MEASUREMENT_ID: ${{ vars.PUBLIC_GA_MEASUREMENT_ID }} + run: pnpm --filter @open-design/landing-page build:static + + - name: Verify zero external JavaScript + run: | + node <<'NODE' + const { readFileSync } = require('node:fs'); + const html = readFileSync('apps/landing-page/out/index.html', 'utf8'); + const forbidden = [ + /]*\bsrc=/i, + /type=["']module["']/i, + /\/_astro\/[^"'<>\s]+\.js/i, + ]; + for (const pattern of forbidden) { + if (pattern.test(html)) { + console.error(`Unexpected client JavaScript matched ${pattern}`); + process.exit(1); + } + } + NODE + + - name: Verify Cloudflare image resizing URLs + run: | + node <<'NODE' + const { readFileSync } = require('node:fs'); + const html = readFileSync('apps/landing-page/out/index.html', 'utf8'); + const resizedUrls = html.match(/https:\/\/static\.open-design\.ai\/cdn-cgi\/image\//g) ?? []; + if (resizedUrls.length < 16) { + console.error(`Expected at least 16 Cloudflare resized image URLs, found ${resizedUrls.length}`); + process.exit(1); + } + if (/(?:src|content)=["']\/assets\/[A-Za-z0-9_.-]+\.png/.test(html)) { + console.error('Found local /assets/*.png image reference in generated landing HTML.'); + process.exit(1); + } + NODE + + # `--branch=main` IS the Cloudflare Pages production branch, so this + # publishes to the production domain (open-design.ai). + - name: Deploy to Cloudflare Pages (production) + uses: cloudflare/wrangler-action@v3 + with: + apiToken: ${{ secrets.CLOUDFLARE_API_TOKEN }} + accountId: ${{ secrets.CLOUDFLARE_ACCOUNT_ID }} + workingDirectory: apps/landing-page + packageManager: npm + command: > + pages deploy out + --project-name=open-design-landing + --branch=main diff --git a/.github/workflows/landing-page-deploy.yml b/.github/workflows/landing-page-staging.yml similarity index 75% rename from .github/workflows/landing-page-deploy.yml rename to .github/workflows/landing-page-staging.yml index 078ec94a4..0191accdd 100644 --- a/.github/workflows/landing-page-deploy.yml +++ b/.github/workflows/landing-page-staging.yml @@ -1,4 +1,11 @@ -name: landing-page-deploy +name: landing-page-staging + +# Pushes to `main` deploy the landing page to the STAGING Cloudflare Pages +# project (`open-design-landing-staging`, served at staging.open-design.ai). +# Production (the separate `open-design-landing` project β†’ open-design.ai) is +# never named here β€” it is reached exclusively through the manual +# `landing-page-production` workflow. This is the safety gate: project +# separation means a merge to main can only ever change staging. on: push: @@ -6,7 +13,8 @@ on: - main paths: # Workflow files - - .github/workflows/landing-page-deploy.yml + - .github/workflows/landing-page-staging.yml + - .github/workflows/landing-page-production.yml - .github/workflows/landing-page-ci.yml # Landing page sources - apps/landing-page/** @@ -31,15 +39,18 @@ permissions: deployments: write concurrency: - group: landing-page-deploy-${{ github.ref }} + group: landing-page-staging-${{ github.ref }} cancel-in-progress: true jobs: deploy: - name: Deploy landing page + name: Deploy landing page to staging if: github.repository == 'nexu-io/open-design' runs-on: ubuntu-latest timeout-minutes: 20 + environment: + name: landing-staging + url: https://staging.open-design.ai steps: - name: Checkout @@ -95,9 +106,15 @@ jobs: - name: Generate skill + template previews run: pnpm --filter @open-design/landing-page previews + # No PUBLIC_GA_MEASUREMENT_ID on staging: leaving it unset omits the + # Google Analytics tag entirely, so test-environment traffic does not + # pollute the production GA property. OD_LANDING_NOINDEX=1 makes every + # page emit so the staging mirror + # stays out of search engines. Only `landing-page-production` builds + # without these (GA on, indexable). - name: Build landing page env: - PUBLIC_GA_MEASUREMENT_ID: ${{ vars.PUBLIC_GA_MEASUREMENT_ID }} + OD_LANDING_NOINDEX: '1' run: pnpm --filter @open-design/landing-page build:static - name: Verify zero external JavaScript @@ -134,7 +151,11 @@ jobs: } NODE - - name: Deploy to Cloudflare Pages + # Deploys to the dedicated STAGING project. `--branch=main` is that + # project's production branch, so it serves at its custom domain + # staging.open-design.ai. The production project (open-design-landing) + # is a different project and is never named here. + - name: Deploy to Cloudflare Pages (staging) uses: cloudflare/wrangler-action@v3 with: apiToken: ${{ secrets.CLOUDFLARE_API_TOKEN }} @@ -143,5 +164,5 @@ jobs: packageManager: npm command: > pages deploy out - --project-name=open-design-landing - --branch=${{ github.ref_name }} + --project-name=open-design-landing-staging + --branch=main diff --git a/apps/landing-page/AGENTS.md b/apps/landing-page/AGENTS.md index cb77af552..ac9cc29eb 100644 --- a/apps/landing-page/AGENTS.md +++ b/apps/landing-page/AGENTS.md @@ -90,10 +90,30 @@ Tightly coupled with: upstream Markdown (e.g., `guizang-ppt`) doesn't break the build when an author uses a slightly different `od:` key. -## Auto-deploy contract +## Deploy contract (staging β†’ manual production) -`.github/workflows/landing-page-deploy.yml` runs on push to `main` -when **any** of these change: +Deploys are split across **two Cloudflare Pages projects** so a merge to +`main` can never publish to the live site on its own: + +- Production project `open-design-landing` β†’ `open-design.ai`. +- Staging project `open-design-landing-staging` β†’ `staging.open-design.ai`. + +The safety gate is project separation: only the manual production workflow +ever names the production project. + +- `.github/workflows/landing-page-staging.yml` runs on push to `main` and + deploys to the **staging project** (`open-design-landing-staging`, + `staging.open-design.ai`). +- `.github/workflows/landing-page-production.yml` is **manual** + (`workflow_dispatch`) and is the only workflow that names the production + project (`open-design-landing`, `open-design.ai`). Gate it with required + reviewers on the GitHub `production` environment. +- `.github/workflows/landing-page-ci.yml` runs on PRs: it validates the build + and, for same-repo branches, deploys a per-PR preview into the staging + project (`--branch=pr-` β†’ + `pr-.open-design-landing-staging.pages.dev`) and comments the URL. + +The staging workflow triggers when **any** of these change: - `apps/landing-page/**` - `design-templates/open-design-landing/**` @@ -102,12 +122,12 @@ when **any** of these change: - `craft/**` - `templates/**` - `package.json`, `pnpm-lock.yaml`, `pnpm-workspace.yaml` -- the workflow file itself +- the workflow files themselves -A push that only edits a SKILL.md MUST trigger this workflow β€” if it -doesn't, the `paths:` filter has drifted from the content-collection -glob and the published site will fall behind silently. Treat that as -a regression, not a feature. +A push that only edits a SKILL.md MUST trigger the staging workflow β€” if it +doesn't, the `paths:` filter has drifted from the content-collection glob and +the staged site will fall behind silently. Treat that as a regression, not a +feature. ## Common commands diff --git a/apps/landing-page/app/_components/seo-head.astro b/apps/landing-page/app/_components/seo-head.astro index 54b718f1f..6fb573f1d 100644 --- a/apps/landing-page/app/_components/seo-head.astro +++ b/apps/landing-page/app/_components/seo-head.astro @@ -55,6 +55,18 @@ export interface SeoHeadProps { const props = Astro.props as SeoHeadProps; +// Staging / PR-preview builds set OD_LANDING_NOINDEX=1 so the mirror at +// staging.open-design.ai (exposed via certificate-transparency logs) is kept +// out of the search index. We emit `noindex` rather than a robots.txt +// `Disallow` so crawlers can still fetch the page and read both this tag and +// the canonical (which points at the production origin). Production builds +// leave the flag unset and stay fully indexable. +// +// `__OD_LANDING_NOINDEX__` is a compile-time constant injected by +// `vite.define` in astro.config.ts β€” `.astro` frontmatter is transformed by +// Vite and cannot read process.env directly. +const noindex = __OD_LANDING_NOINDEX__; + const SITE_NAME = 'Open Design'; const TAGLINE = 'Design with the agent already on your laptop.'; const isArticle = props.kind === 'article'; @@ -149,6 +161,7 @@ const blogJsonLd = {fullTitle} +{noindex && } {alternateLinks.map((entry) => ( diff --git a/apps/landing-page/app/env.d.ts b/apps/landing-page/app/env.d.ts index e16c13c69..7577e0579 100644 --- a/apps/landing-page/app/env.d.ts +++ b/apps/landing-page/app/env.d.ts @@ -1 +1,5 @@ /// + +// Compile-time constant injected by `vite.define` in astro.config.ts. True on +// staging / PR-preview builds (OD_LANDING_NOINDEX=1), false in production. +declare const __OD_LANDING_NOINDEX__: boolean; diff --git a/apps/landing-page/astro.config.ts b/apps/landing-page/astro.config.ts index bcda268ae..c995623c4 100644 --- a/apps/landing-page/astro.config.ts +++ b/apps/landing-page/astro.config.ts @@ -1,8 +1,9 @@ import sitemap, { type SitemapItem } from '@astrojs/sitemap'; -import { readFileSync, readdirSync } from 'node:fs'; +import { appendFileSync, readFileSync, readdirSync } from 'node:fs'; import { join } from 'node:path'; import type { AstroUserConfig } from 'astro'; import { defineConfig } from 'astro/config'; +import type { AstroIntegration } from 'astro'; import { DEFAULT_LOCALE, LANDING_LOCALES, @@ -109,6 +110,34 @@ const editorialPaperTheme: ShikiThemeObject = { // builds (Cloudflare Pages preview deployments, local previews on a // different host) can stamp their own URL without forking the config. const site = process.env.OD_LANDING_SITE ?? 'https://open-design.ai'; +// Staging / PR-preview builds set OD_LANDING_NOINDEX=1. Resolved here (config +// runs in Node and can read process.env) and inlined into components as the +// compile-time constant `__OD_LANDING_NOINDEX__` via vite.define below β€” +// `.astro` frontmatter is transformed by Vite and cannot read process.env. +const landingNoindex = process.env.OD_LANDING_NOINDEX === '1'; + +// Staging / PR-preview only: append a catch-all `X-Robots-Tag: noindex` to the +// Cloudflare Pages `_headers` so EVERY response stays out of search indexes β€” +// including the React-rendered homepage and `og.astro`, which build their own +// and don't go through SeoHead (whose covers HTML pages). +// Production builds (flag unset) leave `_headers` untouched. +const noindexHeaders: AstroIntegration = { + name: 'staging-noindex-headers', + hooks: { + 'astro:build:done': () => { + if (!landingNoindex) return; + // `out/_headers` already exists (copied verbatim from public/_headers + // during the build). Append a catch-all noindex header. Path is built + // from the config dir + outDir rather than the hook's `dir` URL to + // avoid any URL-resolution ambiguity. + appendFileSync( + join(import.meta.dirname, 'out', '_headers'), + '\n# Staging / preview mirror β€” keep out of search indexes.\n/*\n X-Robots-Tag: noindex, nofollow\n', + ); + }, + }, +}; + const sitemapLocales = Object.fromEntries( LANDING_LOCALES.map((locale) => [locale.code, locale.htmlLang]), ); @@ -137,6 +166,11 @@ export default defineConfig({ srcDir: './app', outDir: './out', trailingSlash: 'always', + vite: { + define: { + __OD_LANDING_NOINDEX__: JSON.stringify(landingNoindex), + }, + }, build: { // Inline every emitted stylesheet directly into the HTML . // Trade-off: HTML pages grow by ~10-15KB (already Brotli-compressed @@ -159,6 +193,7 @@ export default defineConfig({ }, }, integrations: [ + noindexHeaders, sitemap({ i18n: { defaultLocale: DEFAULT_LOCALE, diff --git a/apps/landing-page/scripts/blog-indexing/verify-readiness.ts b/apps/landing-page/scripts/blog-indexing/verify-readiness.ts index f05307d6a..32b16cf97 100644 --- a/apps/landing-page/scripts/blog-indexing/verify-readiness.ts +++ b/apps/landing-page/scripts/blog-indexing/verify-readiness.ts @@ -8,7 +8,7 @@ * - page is present in sitemap output * * Polls each URL with a short backoff so the script can be invoked - * immediately after `landing-page-deploy` completes (Cloudflare Pages + * immediately after `landing-page-production` completes (Cloudflare Pages * propagation usually < 60 s but not guaranteed). * * Usage: tsx verify-readiness.ts --urls [--out file.json] [--timeout-ms 180000] diff --git a/docs/blog-indexing-automation.md b/docs/blog-indexing-automation.md index 7a27e4b41..dd0f518b1 100644 --- a/docs/blog-indexing-automation.md +++ b/docs/blog-indexing-automation.md @@ -14,7 +14,7 @@ doc is its concrete implementation in `nexu-io/open-design`. | Trigger | Job | Outcome | |---|---|---| | `landing-page-ci` | `lint-blog-seo.ts` + `check-blog-url-changes.ts` | Changed posts are checked for frontmatter, internal/external links, rendered canonical/JSON-LD/OG metadata, and slug delete/rename redirects before they can merge. | -| `landing-page-deploy` finishes successfully on `main` | `blog-indexing-on-deploy.yml` | New blog URLs are detected, verified ready, submitted to IndexNow, the sitemap-index is re-submitted to GSC, baseline URL Inspection is captured, and baseline Search Analytics is queried. | +| `landing-page-production` promotion finishes successfully | `blog-indexing-on-deploy.yml` | New blog URLs are detected, verified ready, submitted to IndexNow, the sitemap-index is re-submitted to GSC, baseline URL Inspection is captured, and baseline Search Analytics is queried. Staging deploys (`landing-page-staging`) never trigger this. | | Daily `cron: 0 2 * * *` | `blog-indexing-monitor.yml` | Every blog post in the T+1 / T+3 / T+7 / T+14 window is re-inspected; GSC Search Analytics is refreshed; stall and low-traffic issues are opened/refreshed when needed. | | Daily `cron: 0 2 * * *` (10:00 Asia/Shanghai) | `blog-3day-report.yml` | T-3 cohort + 30-day rolling cohort traffic digest written to `docs/blog-traffic-digest.md` via the `automation/blog-traffic-digest` PR, with an optional Feishu group push. Read-only against GSC. | | Manual `workflow_dispatch` | `blog-indexing-monitor.yml` | Maintainers can dry-run or explicitly publish a token-gated dev.to/Hashnode cross-post with canonical URL pointing back to Open Design. | @@ -58,10 +58,19 @@ content / SEO issue. ## Architecture +Because production is a manual promotion that can bundle several merged +posts, `detect-changed-urls` diffs from the `blog-indexed-prod` git tag (the +last commit this workflow successfully indexed) rather than `HEAD^`. The tag +is force-advanced to the deployed commit at the end of a successful run, so +the next promotion picks up exactly the posts merged in between. On the very +first run the tag does not exist yet and the workflow bootstraps from `HEAD^` +β€” create the tag manually (`git tag blog-indexed-prod ; git push origin +blog-indexed-prod`) if an initial multi-commit backfill is needed. + ``` -landing-page-deploy ──success──▢ blog-indexing-on-deploy +landing-page-production ──success──▢ blog-indexing-on-deploy β”‚ - detect-changed-urls + detect-changed-urls (base = blog-indexed-prod tag) β”‚ verify-readiness (200 / canonical / sitemap) β”‚ @@ -224,7 +233,7 @@ The expected steady state: - Renames are handled as both a redirect requirement for the old slug and a newly deployed URL for the destination slug, so the new page is included in the post-deploy readiness and baseline inspection flow. -- New post ships β†’ `landing-page-deploy` runs β†’ `blog-indexing-on-deploy` +- New post ships β†’ `landing-page-production` promotion runs β†’ `blog-indexing-on-deploy` runs β†’ IndexNow is called, GSC sitemap is submitted, and the bot PR opens with the baseline verdict plus any available 7d/28d traffic metrics.