mirror of
https://github.com/nexu-io/open-design.git
synced 2026-06-01 03:14:35 +07:00
fix(plugins): stop recommending raw publish CLIs from authoring summary (#2380)
QA repro (transcript shared on PR #2363): the agent finishes the Home "Create plugin" → plugin-authoring chip flow, validates / packs / installs the generated plugin, then in its summary turn freeform- recommends shell commands like: od plugin publish ./generated-plugin --to open-design cd generated-plugin && git init && git add . && git commit -m "..." gh repo create lefarcen/<name> --public --source=. --push Two things go wrong: 1. The summary recreates the exact flows the plugin-folder card buttons already drive end-to-end (PR #2363 wired the buttons through the right gh + git sequences with auth gates, jq fallback, and retry rules baked in). The freeform suggestions drop those guarantees — they're the source of bug #2332 where `od plugin publish --to open-design` was the recommended action even though it produces an issue URL rather than creating a public repo. 2. The prompt explicitly hinted at `od plugin publish` in step 3 ("Then run or prepare the CLI path: ... and `od plugin publish` when the user is ready to open a registry PR"). The agent dutifully repeated that as the next step, even though the publish work belongs to the plugin-folder card button prompts now. Rewrites `PLUGIN_AUTHORING_PROMPT_TEMPLATE` to: * Keep the original scaffolding instructions (generated-plugin/ + SKILL.md + open-design.json + plugin.repo + inputs). * Replace the loose "od plugin whoami/login through gh, and od plugin publish when the user is ready" sentence with a precise local- validation chain: `od plugin validate` → `od plugin pack` → `od plugin install --source <abs-path>`. * Tell the agent to STOP after the summary, and explicitly ban follow-up suggestions of `od plugin publish`, `od plugin publish --to open-design`, `gh repo create`, `git init` / `git remote add` / `git push`. The ban names the workarounds it has actually emitted in QA transcripts. * Point the user at the plugin-folder card buttons (Add to My plugins / Publish repo / Open Design PR) and call out that those prompts encode the auth gates and fallback rules the summary suggestions would skip. * Same jq guidance carried over from PR #2363: do not assume the standalone `jq` binary, prefer the built-in Read tool / cat / node -e, and disambiguate from `gh ... --jq` (which is fine because gh ships its own embedded library). Tests: * `apps/web/tests/components/home-hero/plugin-authoring-prompt.test.ts` (new) — nine assertions covering: goal-placeholder interpolation, generated-plugin scaffolding mentions, local validation chain, the follow-up CLI ban list (`od plugin publish --to open-design`, `gh repo create`, `git push`), the plugin-folder card hand-off, the jq fallback + gh-flag carve-out, and the round-trip helpers (`buildPluginAuthoringInputs` / `buildPluginAuthoringPromptForInputs` / `createPluginAuthoringHandoff`). Validation: * `pnpm --filter @open-design/web exec vitest run tests/components/home-hero/plugin-authoring-prompt.test.ts` → 9/9 passed * `pnpm --filter @open-design/web exec vitest run tests/components/HomeView.prefill.test.tsx` → 11/11 passed (no regression in the adjacent HomeView prompt-prefill suite; that test references the template literal directly so a silent shape change would surface here) Related: PR #2363 fixes the same source-of-bug shape on the plugin-folder card "Publish repo" button. That PR is the canonical home for the publish flow; this PR makes sure the authoring summary hands off to it rather than re-implementing it inline.
This commit is contained in:
parent
c617e30e27
commit
d03b8fd095
2 changed files with 130 additions and 3 deletions
|
|
@ -27,12 +27,21 @@ export const PLUGIN_AUTHORING_PROMPT_TEMPLATE = [
|
|||
'',
|
||||
'Run the agent-assisted plugin authoring flow end to end. Follow docs/plugins-spec.md and produce a folder named generated-plugin with:',
|
||||
'- SKILL.md describing the agent behavior and workflow',
|
||||
'- open-design.json with valid metadata, vendor/plugin-name naming when publishing, plugin.repo, mode, task kind, inputs, and any pipeline/context references',
|
||||
'- open-design.json with valid metadata: specVersion, name, version, description, plugin.repo (use the `https://github.com/<vendor>/<plugin-name>` shape), mode, task kind, inputs, plus any pipeline / context references the workflow needs',
|
||||
'- optional examples/ and assets/ when useful',
|
||||
'',
|
||||
'Then run or prepare the CLI path: od plugin validate, od plugin pack, local install/run validation, od plugin whoami/login through gh, and od plugin publish when the user is ready to open a registry PR.',
|
||||
'Validate the plugin locally before reporting: run `od plugin validate` on the folder, then `od plugin pack` for a tarball, then `od plugin install --source <absolute-folder-path>` to confirm the install path works.',
|
||||
'',
|
||||
'When finished, summarize files created, validation status, local install/run status, pack output, and the exact publish command or PR next step. End by clearly offering the next actions: Add to My plugins, Publish repo, or Open Design PR.',
|
||||
'When the work above is done, write a single summary turn covering: files created, `od plugin validate` status, local install / run status, and `od plugin pack` output. Then STOP.',
|
||||
'',
|
||||
'**Do NOT** suggest follow-up CLI commands such as `od plugin publish`, `od plugin publish --to open-design`, `gh repo create`, `git init` / `git remote add` / `git push`, or any other publish / repo wiring. The plugin-folder card under Design Files already exposes three buttons whose prompts drive those flows end-to-end with the right auth gates, fallbacks, and retry rules baked in:',
|
||||
'- **Add to My plugins** — already satisfied by this turn\'s `od plugin install --source` step.',
|
||||
'- **Publish repo** — creates / updates the author\'s `plugin.repo` GitHub repo through a gh + git sequence the agent is told exactly how to run.',
|
||||
'- **Open Design PR** — opens a draft PR against `nexu-io/open-design` for the community catalog.',
|
||||
'',
|
||||
'Point the user at whichever button they want next; do NOT recreate those flows as freeform shell suggestions in this summary. Recreating them drifts from the button prompts\' guarantees and is the source of the bug that closed #2332.',
|
||||
'',
|
||||
'**Do NOT** assume the standalone `jq` binary is installed (it is not part of the OD agent runtime baseline and is missing from default macOS / Windows shells). When you need to read the manifest, prefer your built-in file-reading tool, then `cat generated-plugin/open-design.json` followed by manual JSON parsing, then `node -e \'console.log(JSON.parse(require("fs").readFileSync("generated-plugin/open-design.json","utf8")))\'`. The `gh ... --jq` flag is fine because gh ships its own embedded library; the brew-installed standalone `jq` is NOT.',
|
||||
].join('\n');
|
||||
|
||||
export const PLUGIN_AUTHORING_PROMPT = buildPluginAuthoringPrompt(PLUGIN_AUTHORING_DEFAULT_GOAL);
|
||||
|
|
|
|||
|
|
@ -0,0 +1,118 @@
|
|||
import { describe, expect, it } from 'vitest';
|
||||
|
||||
import {
|
||||
PLUGIN_AUTHORING_DEFAULT_GOAL,
|
||||
PLUGIN_AUTHORING_GOAL_INPUT,
|
||||
PLUGIN_AUTHORING_PROMPT,
|
||||
PLUGIN_AUTHORING_PROMPT_TEMPLATE,
|
||||
buildPluginAuthoringPrompt,
|
||||
buildPluginAuthoringInputs,
|
||||
buildPluginAuthoringPromptForInputs,
|
||||
createPluginAuthoringHandoff,
|
||||
} from '../../../src/components/home-hero/plugin-authoring';
|
||||
|
||||
// The Home "Create plugin" chip sends this prompt as the project's first
|
||||
// user turn. When QA exercised it (issue #2332 transcript), the agent's
|
||||
// summary turn freeform-recommended `od plugin publish --to open-design`
|
||||
// and `gh repo create lefarcen/<name>` — recreating the exact flows the
|
||||
// plugin-folder card buttons already own. The button prompts (PR #2363)
|
||||
// encode auth gates, jq fallback, retry rules; agent summaries that
|
||||
// duplicate them as raw shell commands drift from those guarantees and
|
||||
// re-open the same bugs. These tests lock the rewritten prompt's
|
||||
// guard-rails so a future prose edit can't reintroduce the freeform
|
||||
// CLI suggestions.
|
||||
|
||||
describe('PLUGIN_AUTHORING_PROMPT_TEMPLATE', () => {
|
||||
it('keeps the goal placeholder so the template still interpolates', () => {
|
||||
expect(PLUGIN_AUTHORING_PROMPT_TEMPLATE).toContain(`{{${PLUGIN_AUTHORING_GOAL_INPUT}}}`);
|
||||
expect(buildPluginAuthoringPrompt('a SaaS pitch deck workflow')).toContain(
|
||||
'a SaaS pitch deck workflow',
|
||||
);
|
||||
expect(PLUGIN_AUTHORING_PROMPT).toContain(PLUGIN_AUTHORING_DEFAULT_GOAL);
|
||||
});
|
||||
|
||||
it('still asks the agent to scaffold generated-plugin with SKILL.md + manifest', () => {
|
||||
expect(PLUGIN_AUTHORING_PROMPT_TEMPLATE).toContain('generated-plugin');
|
||||
expect(PLUGIN_AUTHORING_PROMPT_TEMPLATE).toContain('SKILL.md');
|
||||
expect(PLUGIN_AUTHORING_PROMPT_TEMPLATE).toContain('open-design.json');
|
||||
expect(PLUGIN_AUTHORING_PROMPT_TEMPLATE).toContain('plugin.repo');
|
||||
});
|
||||
|
||||
it('still drives the local validation chain', () => {
|
||||
expect(PLUGIN_AUTHORING_PROMPT_TEMPLATE).toContain('od plugin validate');
|
||||
expect(PLUGIN_AUTHORING_PROMPT_TEMPLATE).toContain('od plugin pack');
|
||||
expect(PLUGIN_AUTHORING_PROMPT_TEMPLATE).toContain('od plugin install --source');
|
||||
});
|
||||
|
||||
it('bans freeform publish / repo CLI suggestions in the summary turn', () => {
|
||||
// The agent transcript in #2332 had the agent recommending
|
||||
// `od plugin publish --to open-design`, `gh repo create
|
||||
// lefarcen/<name>`, and `git init && git push` in its summary —
|
||||
// recreating the exact flows the plugin-folder card buttons own.
|
||||
// The ban list must name those workarounds explicitly so the agent
|
||||
// can't drift back into them.
|
||||
expect(PLUGIN_AUTHORING_PROMPT_TEMPLATE).toMatch(
|
||||
/Do NOT.*suggest follow-up CLI commands/i,
|
||||
);
|
||||
expect(PLUGIN_AUTHORING_PROMPT_TEMPLATE).toContain('od plugin publish --to open-design');
|
||||
expect(PLUGIN_AUTHORING_PROMPT_TEMPLATE).toContain('gh repo create');
|
||||
expect(PLUGIN_AUTHORING_PROMPT_TEMPLATE).toContain('git push');
|
||||
});
|
||||
|
||||
it('points the user at the plugin-folder card buttons instead', () => {
|
||||
expect(PLUGIN_AUTHORING_PROMPT_TEMPLATE).toContain('Add to My plugins');
|
||||
expect(PLUGIN_AUTHORING_PROMPT_TEMPLATE).toContain('Publish repo');
|
||||
expect(PLUGIN_AUTHORING_PROMPT_TEMPLATE).toContain('Open Design PR');
|
||||
expect(PLUGIN_AUTHORING_PROMPT_TEMPLATE).toMatch(
|
||||
/Point the user at whichever button|Tell the user to click whichever button/i,
|
||||
);
|
||||
});
|
||||
|
||||
it('warns against assuming standalone jq is installed', () => {
|
||||
// Same jq-fallback lesson as PR #2363 — agent reaches for jq first
|
||||
// by training-distribution default. The prompt must list portable
|
||||
// alternatives AND keep gh's --jq flag exempt.
|
||||
// Note `Do NOT\*\* ` matches the markdown-bolded `**Do NOT**` that
|
||||
// sits in the prompt. The bolding is intentional emphasis, so the
|
||||
// regex tolerates the `**` markers between NOT and the rest of the
|
||||
// sentence.
|
||||
expect(PLUGIN_AUTHORING_PROMPT_TEMPLATE).toMatch(
|
||||
/Do NOT\W*assume the standalone `jq` binary is installed/i,
|
||||
);
|
||||
expect(PLUGIN_AUTHORING_PROMPT_TEMPLATE).toMatch(/cat .*open-design\.json/);
|
||||
expect(PLUGIN_AUTHORING_PROMPT_TEMPLATE).toContain('node -e');
|
||||
expect(PLUGIN_AUTHORING_PROMPT_TEMPLATE).toMatch(/`gh \.\.\. --jq` flag is fine|gh ships its own embedded library/i);
|
||||
});
|
||||
});
|
||||
|
||||
describe('buildPluginAuthoringInputs / buildPluginAuthoringPromptForInputs', () => {
|
||||
it('round-trips a user-provided goal through the inputs helper', () => {
|
||||
const inputs = buildPluginAuthoringInputs('outline deck from a brief');
|
||||
expect(inputs[PLUGIN_AUTHORING_GOAL_INPUT]).toBe('outline deck from a brief');
|
||||
const prompt = buildPluginAuthoringPromptForInputs(inputs);
|
||||
expect(prompt).toContain('outline deck from a brief');
|
||||
});
|
||||
|
||||
it('falls back to the default goal when the input is missing or blank', () => {
|
||||
expect(buildPluginAuthoringInputs(undefined)[PLUGIN_AUTHORING_GOAL_INPUT]).toBe(
|
||||
PLUGIN_AUTHORING_DEFAULT_GOAL,
|
||||
);
|
||||
expect(buildPluginAuthoringInputs(' ')[PLUGIN_AUTHORING_GOAL_INPUT]).toBe(
|
||||
PLUGIN_AUTHORING_DEFAULT_GOAL,
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
describe('createPluginAuthoringHandoff', () => {
|
||||
it('returns a plugin-authoring handoff with the rewritten prompt', () => {
|
||||
const handoff = createPluginAuthoringHandoff(1, 'a slide outline workflow');
|
||||
expect(handoff.source).toBe('plugin-authoring');
|
||||
if (handoff.source !== 'plugin-authoring') return;
|
||||
expect(handoff.goal).toBe('a slide outline workflow');
|
||||
expect(handoff.prompt).toContain('a slide outline workflow');
|
||||
// The handoff must carry the latest template so HomeView's
|
||||
// replacement-confirmation logic (`queueAuthoringChipId === 'create-plugin'`)
|
||||
// sends the rewritten copy and not a cached older string.
|
||||
expect(handoff.queryTemplate).toBe(PLUGIN_AUTHORING_PROMPT_TEMPLATE);
|
||||
});
|
||||
});
|
||||
Loading…
Reference in a new issue