fix(plugins): stop recommending raw publish CLIs from authoring summary (#2380)

QA repro (transcript shared on PR #2363): the agent finishes the
Home "Create plugin" → plugin-authoring chip flow, validates / packs
/ installs the generated plugin, then in its summary turn freeform-
recommends shell commands like:

  od plugin publish ./generated-plugin --to open-design
  cd generated-plugin && git init && git add . && git commit -m "..."
  gh repo create lefarcen/<name> --public --source=. --push

Two things go wrong:

1. The summary recreates the exact flows the plugin-folder card buttons
   already drive end-to-end (PR #2363 wired the buttons through the
   right gh + git sequences with auth gates, jq fallback, and retry
   rules baked in). The freeform suggestions drop those guarantees —
   they're the source of bug #2332 where `od plugin publish --to
   open-design` was the recommended action even though it produces an
   issue URL rather than creating a public repo.

2. The prompt explicitly hinted at `od plugin publish` in step 3 ("Then
   run or prepare the CLI path: ... and `od plugin publish` when the
   user is ready to open a registry PR"). The agent dutifully repeated
   that as the next step, even though the publish work belongs to the
   plugin-folder card button prompts now.

Rewrites `PLUGIN_AUTHORING_PROMPT_TEMPLATE` to:

* Keep the original scaffolding instructions (generated-plugin/ +
  SKILL.md + open-design.json + plugin.repo + inputs).
* Replace the loose "od plugin whoami/login through gh, and od plugin
  publish when the user is ready" sentence with a precise local-
  validation chain: `od plugin validate` → `od plugin pack` →
  `od plugin install --source <abs-path>`.
* Tell the agent to STOP after the summary, and explicitly ban
  follow-up suggestions of `od plugin publish`, `od plugin publish
  --to open-design`, `gh repo create`, `git init` / `git remote add` /
  `git push`. The ban names the workarounds it has actually emitted
  in QA transcripts.
* Point the user at the plugin-folder card buttons (Add to My plugins
  / Publish repo / Open Design PR) and call out that those prompts
  encode the auth gates and fallback rules the summary suggestions
  would skip.
* Same jq guidance carried over from PR #2363: do not assume the
  standalone `jq` binary, prefer the built-in Read tool / cat / node
  -e, and disambiguate from `gh ... --jq` (which is fine because gh
  ships its own embedded library).

Tests:

* `apps/web/tests/components/home-hero/plugin-authoring-prompt.test.ts`
  (new) — nine assertions covering: goal-placeholder interpolation,
  generated-plugin scaffolding mentions, local validation chain, the
  follow-up CLI ban list (`od plugin publish --to open-design`,
  `gh repo create`, `git push`), the plugin-folder card hand-off, the
  jq fallback + gh-flag carve-out, and the round-trip helpers
  (`buildPluginAuthoringInputs` / `buildPluginAuthoringPromptForInputs`
  / `createPluginAuthoringHandoff`).

Validation:

* `pnpm --filter @open-design/web exec vitest run tests/components/home-hero/plugin-authoring-prompt.test.ts`
  → 9/9 passed
* `pnpm --filter @open-design/web exec vitest run tests/components/HomeView.prefill.test.tsx`
  → 11/11 passed (no regression in the adjacent HomeView prompt-prefill
  suite; that test references the template literal directly so a
  silent shape change would surface here)

Related: PR #2363 fixes the same source-of-bug shape on the
plugin-folder card "Publish repo" button. That PR is the canonical
home for the publish flow; this PR makes sure the authoring summary
hands off to it rather than re-implementing it inline.
This commit is contained in:
lefarcen 2026-05-20 15:39:42 +08:00 committed by GitHub
parent c617e30e27
commit d03b8fd095
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
2 changed files with 130 additions and 3 deletions

View file

@ -27,12 +27,21 @@ export const PLUGIN_AUTHORING_PROMPT_TEMPLATE = [
'',
'Run the agent-assisted plugin authoring flow end to end. Follow docs/plugins-spec.md and produce a folder named generated-plugin with:',
'- SKILL.md describing the agent behavior and workflow',
'- open-design.json with valid metadata, vendor/plugin-name naming when publishing, plugin.repo, mode, task kind, inputs, and any pipeline/context references',
'- open-design.json with valid metadata: specVersion, name, version, description, plugin.repo (use the `https://github.com/<vendor>/<plugin-name>` shape), mode, task kind, inputs, plus any pipeline / context references the workflow needs',
'- optional examples/ and assets/ when useful',
'',
'Then run or prepare the CLI path: od plugin validate, od plugin pack, local install/run validation, od plugin whoami/login through gh, and od plugin publish when the user is ready to open a registry PR.',
'Validate the plugin locally before reporting: run `od plugin validate` on the folder, then `od plugin pack` for a tarball, then `od plugin install --source <absolute-folder-path>` to confirm the install path works.',
'',
'When finished, summarize files created, validation status, local install/run status, pack output, and the exact publish command or PR next step. End by clearly offering the next actions: Add to My plugins, Publish repo, or Open Design PR.',
'When the work above is done, write a single summary turn covering: files created, `od plugin validate` status, local install / run status, and `od plugin pack` output. Then STOP.',
'',
'**Do NOT** suggest follow-up CLI commands such as `od plugin publish`, `od plugin publish --to open-design`, `gh repo create`, `git init` / `git remote add` / `git push`, or any other publish / repo wiring. The plugin-folder card under Design Files already exposes three buttons whose prompts drive those flows end-to-end with the right auth gates, fallbacks, and retry rules baked in:',
'- **Add to My plugins** — already satisfied by this turn\'s `od plugin install --source` step.',
'- **Publish repo** — creates / updates the author\'s `plugin.repo` GitHub repo through a gh + git sequence the agent is told exactly how to run.',
'- **Open Design PR** — opens a draft PR against `nexu-io/open-design` for the community catalog.',
'',
'Point the user at whichever button they want next; do NOT recreate those flows as freeform shell suggestions in this summary. Recreating them drifts from the button prompts\' guarantees and is the source of the bug that closed #2332.',
'',
'**Do NOT** assume the standalone `jq` binary is installed (it is not part of the OD agent runtime baseline and is missing from default macOS / Windows shells). When you need to read the manifest, prefer your built-in file-reading tool, then `cat generated-plugin/open-design.json` followed by manual JSON parsing, then `node -e \'console.log(JSON.parse(require("fs").readFileSync("generated-plugin/open-design.json","utf8")))\'`. The `gh ... --jq` flag is fine because gh ships its own embedded library; the brew-installed standalone `jq` is NOT.',
].join('\n');
export const PLUGIN_AUTHORING_PROMPT = buildPluginAuthoringPrompt(PLUGIN_AUTHORING_DEFAULT_GOAL);

View file

@ -0,0 +1,118 @@
import { describe, expect, it } from 'vitest';
import {
PLUGIN_AUTHORING_DEFAULT_GOAL,
PLUGIN_AUTHORING_GOAL_INPUT,
PLUGIN_AUTHORING_PROMPT,
PLUGIN_AUTHORING_PROMPT_TEMPLATE,
buildPluginAuthoringPrompt,
buildPluginAuthoringInputs,
buildPluginAuthoringPromptForInputs,
createPluginAuthoringHandoff,
} from '../../../src/components/home-hero/plugin-authoring';
// The Home "Create plugin" chip sends this prompt as the project's first
// user turn. When QA exercised it (issue #2332 transcript), the agent's
// summary turn freeform-recommended `od plugin publish --to open-design`
// and `gh repo create lefarcen/<name>` — recreating the exact flows the
// plugin-folder card buttons already own. The button prompts (PR #2363)
// encode auth gates, jq fallback, retry rules; agent summaries that
// duplicate them as raw shell commands drift from those guarantees and
// re-open the same bugs. These tests lock the rewritten prompt's
// guard-rails so a future prose edit can't reintroduce the freeform
// CLI suggestions.
describe('PLUGIN_AUTHORING_PROMPT_TEMPLATE', () => {
it('keeps the goal placeholder so the template still interpolates', () => {
expect(PLUGIN_AUTHORING_PROMPT_TEMPLATE).toContain(`{{${PLUGIN_AUTHORING_GOAL_INPUT}}}`);
expect(buildPluginAuthoringPrompt('a SaaS pitch deck workflow')).toContain(
'a SaaS pitch deck workflow',
);
expect(PLUGIN_AUTHORING_PROMPT).toContain(PLUGIN_AUTHORING_DEFAULT_GOAL);
});
it('still asks the agent to scaffold generated-plugin with SKILL.md + manifest', () => {
expect(PLUGIN_AUTHORING_PROMPT_TEMPLATE).toContain('generated-plugin');
expect(PLUGIN_AUTHORING_PROMPT_TEMPLATE).toContain('SKILL.md');
expect(PLUGIN_AUTHORING_PROMPT_TEMPLATE).toContain('open-design.json');
expect(PLUGIN_AUTHORING_PROMPT_TEMPLATE).toContain('plugin.repo');
});
it('still drives the local validation chain', () => {
expect(PLUGIN_AUTHORING_PROMPT_TEMPLATE).toContain('od plugin validate');
expect(PLUGIN_AUTHORING_PROMPT_TEMPLATE).toContain('od plugin pack');
expect(PLUGIN_AUTHORING_PROMPT_TEMPLATE).toContain('od plugin install --source');
});
it('bans freeform publish / repo CLI suggestions in the summary turn', () => {
// The agent transcript in #2332 had the agent recommending
// `od plugin publish --to open-design`, `gh repo create
// lefarcen/<name>`, and `git init && git push` in its summary —
// recreating the exact flows the plugin-folder card buttons own.
// The ban list must name those workarounds explicitly so the agent
// can't drift back into them.
expect(PLUGIN_AUTHORING_PROMPT_TEMPLATE).toMatch(
/Do NOT.*suggest follow-up CLI commands/i,
);
expect(PLUGIN_AUTHORING_PROMPT_TEMPLATE).toContain('od plugin publish --to open-design');
expect(PLUGIN_AUTHORING_PROMPT_TEMPLATE).toContain('gh repo create');
expect(PLUGIN_AUTHORING_PROMPT_TEMPLATE).toContain('git push');
});
it('points the user at the plugin-folder card buttons instead', () => {
expect(PLUGIN_AUTHORING_PROMPT_TEMPLATE).toContain('Add to My plugins');
expect(PLUGIN_AUTHORING_PROMPT_TEMPLATE).toContain('Publish repo');
expect(PLUGIN_AUTHORING_PROMPT_TEMPLATE).toContain('Open Design PR');
expect(PLUGIN_AUTHORING_PROMPT_TEMPLATE).toMatch(
/Point the user at whichever button|Tell the user to click whichever button/i,
);
});
it('warns against assuming standalone jq is installed', () => {
// Same jq-fallback lesson as PR #2363 — agent reaches for jq first
// by training-distribution default. The prompt must list portable
// alternatives AND keep gh's --jq flag exempt.
// Note `Do NOT\*\* ` matches the markdown-bolded `**Do NOT**` that
// sits in the prompt. The bolding is intentional emphasis, so the
// regex tolerates the `**` markers between NOT and the rest of the
// sentence.
expect(PLUGIN_AUTHORING_PROMPT_TEMPLATE).toMatch(
/Do NOT\W*assume the standalone `jq` binary is installed/i,
);
expect(PLUGIN_AUTHORING_PROMPT_TEMPLATE).toMatch(/cat .*open-design\.json/);
expect(PLUGIN_AUTHORING_PROMPT_TEMPLATE).toContain('node -e');
expect(PLUGIN_AUTHORING_PROMPT_TEMPLATE).toMatch(/`gh \.\.\. --jq` flag is fine|gh ships its own embedded library/i);
});
});
describe('buildPluginAuthoringInputs / buildPluginAuthoringPromptForInputs', () => {
it('round-trips a user-provided goal through the inputs helper', () => {
const inputs = buildPluginAuthoringInputs('outline deck from a brief');
expect(inputs[PLUGIN_AUTHORING_GOAL_INPUT]).toBe('outline deck from a brief');
const prompt = buildPluginAuthoringPromptForInputs(inputs);
expect(prompt).toContain('outline deck from a brief');
});
it('falls back to the default goal when the input is missing or blank', () => {
expect(buildPluginAuthoringInputs(undefined)[PLUGIN_AUTHORING_GOAL_INPUT]).toBe(
PLUGIN_AUTHORING_DEFAULT_GOAL,
);
expect(buildPluginAuthoringInputs(' ')[PLUGIN_AUTHORING_GOAL_INPUT]).toBe(
PLUGIN_AUTHORING_DEFAULT_GOAL,
);
});
});
describe('createPluginAuthoringHandoff', () => {
it('returns a plugin-authoring handoff with the rewritten prompt', () => {
const handoff = createPluginAuthoringHandoff(1, 'a slide outline workflow');
expect(handoff.source).toBe('plugin-authoring');
if (handoff.source !== 'plugin-authoring') return;
expect(handoff.goal).toBe('a slide outline workflow');
expect(handoff.prompt).toContain('a slide outline workflow');
// The handoff must carry the latest template so HomeView's
// replacement-confirmation logic (`queueAuthoringChipId === 'create-plugin'`)
// sends the rewritten copy and not a cached older string.
expect(handoff.queryTemplate).toBe(PLUGIN_AUTHORING_PROMPT_TEMPLATE);
});
});