open-design/plugins/spec/templates/evals.template.json
pftom a0a3ad1299 feat(plugins): introduce portable plugin specification and authoring kit
- Replaced the `community/` directory with `spec/`, which now serves as the portable plugin specification and authoring kit, including documentation, templates, and examples for contributors.
- Added new files for agent development, contributing guidelines, and a comprehensive README for the plugin spec kit.
- Updated existing documentation to reflect the new structure and clarify authoring rules for plugins.
- Introduced multiple example plugins under the new `spec/examples/` directory, showcasing various use cases and workflows for plugin authors.

This update enhances the clarity and accessibility of the plugin development process, providing a robust framework for contributors and external agents.
2026-05-13 21:24:36 +08:00

26 lines
868 B
JSON

{
"skill_name": "plugin-id",
"evals": [
{
"id": "happy-path",
"prompt": "Create a polished artifact for a realistic user request.",
"expected_output": "A usable artifact plus a concise summary.",
"assertions": [
"The expected output file exists",
"The output directly addresses the user request",
"The output includes no obvious placeholder text",
"The final response names the produced artifact"
]
},
{
"id": "missing-inputs",
"prompt": "Create the artifact, but omit one critical input.",
"expected_output": "The agent asks only for the missing required input or chooses a safe default when the plugin allows it.",
"assertions": [
"The agent does not fabricate external facts",
"The follow-up question is specific and minimal"
]
}
]
}