mirror of
https://github.com/ZSeven-W/openpencil.git
synced 2026-05-31 19:04:29 +07:00
* fix(desktop,web): rebuild Electron dev sync + bitmap dragging fix on v0.7.1 (#99) Re-appliesb046a0dfrom the closed PR #97 now that the base is v0.7.1. Original conflict against v0.7.0 came from the release branch churn — the cherry-pick onto v0.7.1 applies cleanly. Keeps dev-startup, sync-noise, and bitmap-dragging fixes; drops the loopback proxy helper scripts upstream rejected in PR #92. Readiness probe now does direct socket checks inside the existing dev entrypoint; sync hardening stays focused on request diagnostics, backpressure, and drag-time clip-rect correctness. Original commit:b046a0dSupersedes: #97 (closed, head branch deleted) Co-authored-by: Rais <vdcoolzi@gmail.com> * fix(canvas): use ImageFill.url in skia-interaction test The image-backed rectangle fixture in the skia-interaction test used `{ type: 'image', src: '…' }`, but `ImageFill` in pen-types declares the field as `url`. `npx tsc --noEmit` flagged it as TS2352 on the `as PenNode` cast. One-word rename. The squash-merge of #99 captured an earlier snapshot that did not include this fix, so re-apply directly on v0.7.1. * feat(cli): add `op install` / `op uninstall` for openpencil-skill Bundle skill files at build time (scripts/bundle-skill.ts → skill-bundle.json) so users without GitHub access can install directly. Falls back to git clone when the bundle is empty. Supports auto-detection of: Claude Code, Codex, Cursor, Gemini CLI, OpenCode. CI workflows updated to checkout openpencil-skill before cli:compile. * fix(panels): allow reparenting nodes into rectangle in layer panel CONTAINER_TYPES was missing 'rectangle', preventing drag-drop into rectangles even though the data model (ContainerProps) and store (moveNode) both support it. * fix(agent,ai): tool_exec reset + insert_node with after + move_node + CRUD tools - fix(agent): reset StreamingToolExecutor between turns — prevents stale tool_use IDs that caused 400 errors on multi-turn tool calls (MiniMax etc.) - feat(ai): add insert_node "after" parameter — auto-resolves sibling's parent and position for intuitive node insertion - feat(ai): add move_node and insert_node to CRUD tool set - feat(ai): add Chinese keywords (增加/添加/插入) to design intent detection - fix(ai): insert_node uses addNode directly for existing parents instead of streaming pipeline, fixing parent resolution * feat(ai): route CRUD intents to lightweight prompt and tool set CRUD operations (read/update/delete) now get a focused system prompt without design generation instructions, and use getCrudToolDefs() (which includes insert_node and move_node) instead of the full design set. * fix(agent,mcp): submodule update + MCP tool improvements - Update agent-native submodule (tool_exec reset, HTTP error diagnostics) - Improve MCP tool descriptions and parameter schemas - Enhance agent.ts error handling * feat(ai): add PenNode examples to CRUD prompt for complete node generation The CRUD system prompt now includes button and text node examples showing the full structure (fills, children, icons, layout) so models generate complete nodes instead of empty frames. * Update agent-native submodule to commit f9633a8, ensuring compatibility with recent changes and improvements in the agent's functionality. * fix(ci,agent): generate skill-bundle before type check + fix moveNode arg - Add `bun run cli:bundle-skill` step in CI before `tsc --noEmit` so skill-bundle.json exists when type-checking the CLI - Fix moveNode index parameter: default to -1 when undefined * fix(ci): add pen-engine and pen-react to npm publish workflow Insert pen-engine and pen-react in topological order between pen-renderer and pen-mcp so they are published before pen-sdk. * docs: add MIT LICENSE and README to all packages - Add MIT LICENSE to pen-ai-skills, pen-core, pen-engine, pen-figma, pen-mcp, pen-react, pen-renderer, pen-sdk, pen-types - Add README.md to pen-engine, pen-react, pen-mcp, pen-ai-skills * docs: comprehensive package metadata, README, CLAUDE.md, and LICENSE - Add author, license, repository, bugs, homepage to all package.json - Homepage points to each package's own directory on GitHub - Rewrite README for pen-engine, pen-react, pen-mcp, pen-ai-skills with full API docs, usage examples, and feature tables - Add CLAUDE.md to pen-types, pen-core, pen-engine, pen-figma, pen-mcp, pen-react, pen-renderer, pen-sdk - Add MIT LICENSE to all packages - Update root CLAUDE.md with index of all sub-CLAUDE.md files - Fix git URL from nicepkg → ZSeven-W * docs: package metadata, README, LICENSE, and CLAUDE.md for all packages - Add author, license, repository, bugs, homepage to all package.json - Homepage points to each package's own directory on GitHub - Rewrite README for pen-engine, pen-react, pen-mcp, pen-ai-skills with full API docs, usage examples, and feature tables - Add MIT LICENSE to all packages missing it - Add CLAUDE.md to pen-types - Update root CLAUDE.md with index of all sub-CLAUDE.md files - Fix git URL from nicepkg to ZSeven-W * docs: rewrite README for pen-core, pen-figma, pen-renderer, pen-sdk Comprehensive READMEs with full API reference, usage examples, feature tables, and architecture overview for each package. * feat(acp): acpAgents store — persist, hydrate, CRUD actions * feat(acp): pen-acp package + agent settings types + store - pen-acp/types.ts: AcpAgentConfig, AcpAgentInfo, AcpConnectResult, AcpConnectionState - pen-acp/client.ts: connectAcpAgent (local stdio + remote WebSocket), disconnectAcpAgent - pen-acp/event-adapter.ts: acpUpdateToSSE (ACP session/update → SSE events) - agent-settings.ts: AcpAgentConfig type, widen ModelGroup/GroupedModel.provider - agent-settings-store.ts: acpAgents persist/hydrate/CRUD + acpConnectionStatus * fix(acp): remove unused type imports in client.ts * feat(acp): ACP agent settings UI — form, cards, connect/disconnect * feat(acp): add AcpAgentSection to Agents settings tab * refactor(agent): AgentSession as discriminated union (native | acp) * feat(acp): connection manager — connect, disconnect, cleanup * feat(acp): connect/disconnect API route * feat(acp): ACP branch in agent result handler * fix(agent): use NativeAgentSession type for runDelegateMember * feat(acp): ACP prompt SSE stream in agent endpoint * feat(acp): ACP agents in model list + request routing * i18n(acp): add ACP agent translation keys for all 15 locales * fix(i18n): translate ACP keys for all 15 locales + fix missing key references - zh.ts/zh-tw.ts: proper Chinese translations (Agent not translated) - All other locales: translated from English placeholders - Fix acp.add → acp.addAgent, acp.disconnected → acp.notConnected * chore: bump agent-native submodule — surface upstream HTTP errors * feat(acp,build,codegen): comprehensive fixes for ACP integration + prod build ACP agent integration: - Rewrite system prompt to enforce layered design pipeline (get_design_prompt → design_skeleton → design_content → design_refine) for higher quality output - Use correct PenNode field names: content for text, iconFontName for icons - Strict JSON rules to prevent empty-key / trailing-comma / smart-quote errors - Prefer icon_font over path icons (standalone MCP has no hooks registered) - Auto-start MCP server before ACP session (lazy bootstrap) - Auto-reconnect ACP on stale connection (dev server restart scenario) - Auto-approve tool permission requests (trust model: user configured agent) - Use type: 'http' + headers: [] for MCP server config (SDK schema requirement) - Persist ACP connections via globalThis so they survive Vite HMR Build / packaging: - Place agent-native under server/node_modules for Nitro to resolve at runtime - Copy agent_napi.node to napi/ as extraResource - Kill detached MCP server on Electron quit (before-quit + dev SIGINT handlers) - Capture drag-dropped filesystem path via webUtils.getPathForFile so recent files entries are clickable after reopening Codegen: - Compact JSON (no indent) + strip noise fields (id, parentId, default rotation /opacity/visible, layout-managed x/y) to reduce request body size by 60-70% so proxies don't reject with 403 'Request not allowed' MCP batch_design robustness: - splitOperations tracks bracket/quote balance → multi-line JSON now works - Auto-normalize fill/stroke shorthand forms - Collect per-line errors instead of aborting whole batch - Repair empty keys, trailing commas, smart quotes in JSON - Bindless I(...) form supported (auto-generates binding) UI: - ModelDropdown / ChatInput handle ACP model icons (Plug) - Reset streaming state + abort controller on ACP error path - Strip h3 JSON error wrapper so chat shows clean error messages - ACP agent settings form + cards + connect/disconnect * fix(types): resolve TS errors in CI typecheck - acp-connection-manager.ts: correct relative import path (utils/ → src/types) - ai-chat-handlers.ts: cast currentProvider to AIProviderType at design-generator callsites - ai-chat-panel.tsx: explicitly type groups as ModelGroup[] so 'acp' string fits the widened union - acp-agent-settings.tsx: cast window through unknown for Record lookup - electron.d.ts: add getPathForFile to ElectronAPI declaration - builtin-provider-presets.ts: drop now-redundant config.preset !== 'custom' check (handled by early return) - pen-acp/client.ts: cast Writable/Readable.toWeb to typed Streams; coerce nullish agentInfo fields to undefined --------- Co-authored-by: Rais <vdcoolzi@gmail.com>
361 lines
12 KiB
TypeScript
361 lines
12 KiB
TypeScript
import type { AuthLevel } from '@/types/agent';
|
|
|
|
export interface ToolDef {
|
|
name: string;
|
|
description: string;
|
|
level: AuthLevel;
|
|
parameters: Record<string, unknown>;
|
|
}
|
|
|
|
const TOOL_AUTH_MAP: Record<string, AuthLevel> = {
|
|
// read
|
|
batch_get: 'read',
|
|
snapshot_layout: 'read',
|
|
get_selection: 'read',
|
|
get_variables: 'read',
|
|
find_empty_space: 'read',
|
|
get_design_prompt: 'read',
|
|
list_theme_presets: 'read',
|
|
get_design_md: 'read',
|
|
|
|
// create
|
|
plan_layout: 'create',
|
|
batch_insert: 'create',
|
|
insert_node: 'create',
|
|
add_page: 'create',
|
|
duplicate_page: 'create',
|
|
import_svg: 'create',
|
|
copy_node: 'create',
|
|
save_theme_preset: 'create',
|
|
generate_design: 'create',
|
|
|
|
// modify
|
|
update_node: 'modify',
|
|
replace_node: 'modify',
|
|
move_node: 'modify',
|
|
set_variables: 'modify',
|
|
set_themes: 'modify',
|
|
load_theme_preset: 'modify',
|
|
rename_page: 'modify',
|
|
reorder_page: 'modify',
|
|
batch_design: 'modify',
|
|
set_design_md: 'modify',
|
|
export_design_md: 'modify',
|
|
|
|
// delete
|
|
delete_node: 'delete',
|
|
remove_page: 'delete',
|
|
};
|
|
|
|
// ---------------------------------------------------------------------------
|
|
// Intent detection — determines which tools and prompts to load
|
|
// ---------------------------------------------------------------------------
|
|
|
|
export type AgentIntent = 'design' | 'crud';
|
|
|
|
// CJK characters aren't `\w`, so `\b` boundaries silently fail for them.
|
|
// Keep the English list boundary-anchored (avoids `app` matching `approach`,
|
|
// `add` matching `address`, etc.) and run a separate boundary-free pass for
|
|
// CJK keywords.
|
|
const DESIGN_KEYWORDS_EN =
|
|
/\b(design|create|make|build|generate|add|insert|landing|page|screen|app|dashboard|card|hero|navbar|form|layout)\b/i;
|
|
const DESIGN_KEYWORDS_CJK =
|
|
/(设计|创建|生成|画|做一个|新建|增加|添加|加一个|插入|页面|界面|登录|首页|仪表盘|卡片|表单)/;
|
|
|
|
/** Detect whether the user's message is a design intent or a CRUD operation. */
|
|
export function detectAgentIntent(message: string): AgentIntent {
|
|
return DESIGN_KEYWORDS_EN.test(message) || DESIGN_KEYWORDS_CJK.test(message) ? 'design' : 'crud';
|
|
}
|
|
|
|
// ---------------------------------------------------------------------------
|
|
// Tool definitions by intent
|
|
// ---------------------------------------------------------------------------
|
|
|
|
/** CRUD-only tools — lightweight set for read/update/delete/move/insert operations. */
|
|
export function getCrudToolDefs(): ToolDef[] {
|
|
return [
|
|
{
|
|
name: 'batch_get',
|
|
description:
|
|
'Search and read nodes from the document. ALWAYS call this first before update_node or delete_node to find the correct node IDs. ' +
|
|
'With no arguments, returns top-level children (current page structure). Search by type/name patterns or read specific IDs.',
|
|
level: TOOL_AUTH_MAP.batch_get,
|
|
parameters: {
|
|
type: 'object',
|
|
properties: {
|
|
ids: { type: 'array', items: { type: 'string' }, description: 'Node IDs to retrieve' },
|
|
patterns: {
|
|
type: 'array',
|
|
items: { type: 'string' },
|
|
description: 'Search patterns to match',
|
|
},
|
|
},
|
|
},
|
|
},
|
|
{
|
|
name: 'snapshot_layout',
|
|
description:
|
|
'Get a compact layout snapshot of the current page showing node positions and sizes',
|
|
level: TOOL_AUTH_MAP.snapshot_layout,
|
|
parameters: {
|
|
type: 'object',
|
|
properties: {
|
|
pageId: { type: 'string' },
|
|
},
|
|
},
|
|
},
|
|
{
|
|
name: 'insert_node',
|
|
description:
|
|
'Insert a new node into the document tree. Always call snapshot_layout or batch_get first. ' +
|
|
'Use "after" to insert next to a sibling (auto-finds parent and position), or "parent" for explicit placement.',
|
|
level: TOOL_AUTH_MAP.insert_node,
|
|
parameters: {
|
|
type: 'object',
|
|
properties: {
|
|
after: {
|
|
type: 'string',
|
|
description:
|
|
'Insert after this sibling node ID (preferred). Automatically uses the same parent and places the new node right after it.',
|
|
},
|
|
parent: {
|
|
type: ['string', 'null'],
|
|
description:
|
|
'Explicit parent node ID. Use "after" instead when adding next to existing elements.',
|
|
},
|
|
data: {
|
|
type: 'object',
|
|
description: 'PenNode data (type, name, width, height, fills, children, etc.)',
|
|
},
|
|
},
|
|
required: ['data'],
|
|
},
|
|
},
|
|
{
|
|
name: 'update_node',
|
|
description: 'Update properties of an existing node by ID',
|
|
level: TOOL_AUTH_MAP.update_node,
|
|
parameters: {
|
|
type: 'object',
|
|
properties: {
|
|
id: { type: 'string', description: 'Node ID to update' },
|
|
data: { type: 'object', description: 'Properties to update' },
|
|
},
|
|
required: ['id', 'data'],
|
|
},
|
|
},
|
|
{
|
|
name: 'move_node',
|
|
description:
|
|
'Move a node to a different parent container. Use when you need to reparent a node (e.g. move an element into a frame). ' +
|
|
"The node will be placed at the end of the parent's children list by default.",
|
|
level: TOOL_AUTH_MAP.move_node,
|
|
parameters: {
|
|
type: 'object',
|
|
properties: {
|
|
id: { type: 'string', description: 'Node ID to move' },
|
|
parent: { type: 'string', description: 'New parent node ID' },
|
|
index: { type: 'number', description: 'Position index within parent (optional)' },
|
|
},
|
|
required: ['id', 'parent'],
|
|
},
|
|
},
|
|
{
|
|
name: 'delete_node',
|
|
description:
|
|
'Delete a node (and all its children) from the document. ' +
|
|
'Use when the user asks to remove, delete, or clear elements. ' +
|
|
'Always call batch_get first to find the correct node ID before deleting.',
|
|
level: TOOL_AUTH_MAP.delete_node,
|
|
parameters: {
|
|
type: 'object',
|
|
properties: {
|
|
id: { type: 'string', description: 'Node ID to delete' },
|
|
},
|
|
required: ['id'],
|
|
},
|
|
},
|
|
{
|
|
name: 'get_selection',
|
|
description: 'Get the currently selected nodes on the canvas with their full data',
|
|
level: TOOL_AUTH_MAP.get_selection,
|
|
parameters: {
|
|
type: 'object',
|
|
properties: {},
|
|
},
|
|
},
|
|
];
|
|
}
|
|
|
|
/**
|
|
* Design tools — minimal set forcing the model through `generate_design`.
|
|
*
|
|
* Weak models (e.g. MiniMax-M2.7) prefer per-node `insert_node` calls when
|
|
* given the choice, producing scattered output instead of a coherent design.
|
|
* Read tools (`batch_get`, `snapshot_layout`) stay so the model can inspect
|
|
* existing context before generating. CRUD tools live in `getCrudToolDefs()`
|
|
* and are reached via `detectAgentIntent('crud')` for surgical edits.
|
|
*/
|
|
export function getDesignToolDefs(): ToolDef[] {
|
|
return [
|
|
{
|
|
name: 'batch_get',
|
|
description:
|
|
'Search and read nodes from the document. ALWAYS call this first before update_node or delete_node to find the correct node IDs. ' +
|
|
'With no arguments, returns top-level children (current page structure). Search by type/name patterns or read specific IDs.',
|
|
level: TOOL_AUTH_MAP.batch_get,
|
|
parameters: {
|
|
type: 'object',
|
|
properties: {
|
|
ids: { type: 'array', items: { type: 'string' }, description: 'Node IDs to retrieve' },
|
|
patterns: {
|
|
type: 'array',
|
|
items: { type: 'string' },
|
|
description: 'Search patterns to match',
|
|
},
|
|
},
|
|
},
|
|
},
|
|
{
|
|
name: 'snapshot_layout',
|
|
description:
|
|
'Get a compact layout snapshot of the current page showing node positions and sizes',
|
|
level: TOOL_AUTH_MAP.snapshot_layout,
|
|
parameters: {
|
|
type: 'object',
|
|
properties: {
|
|
pageId: { type: 'string' },
|
|
},
|
|
},
|
|
},
|
|
{
|
|
name: 'generate_design',
|
|
description:
|
|
'Generate a complete design on the canvas. Pass a natural language description. The pipeline handles layout, styling, icons, and rendering. Always use this for creating designs.',
|
|
level: TOOL_AUTH_MAP.generate_design,
|
|
parameters: {
|
|
type: 'object',
|
|
properties: {
|
|
prompt: {
|
|
type: 'string',
|
|
description:
|
|
'Natural language description of the design, e.g. "a modern mobile login screen with email, password, login button, and social login"',
|
|
},
|
|
},
|
|
required: ['prompt'],
|
|
},
|
|
},
|
|
];
|
|
}
|
|
|
|
/**
|
|
* Builtin single-agent flows create the frame via `plan_layout` and then
|
|
* insert content with `batch_insert`. Do not expose `generate_design` here,
|
|
* because in builtin mode it only creates the frame and is not a complete
|
|
* design operation.
|
|
*/
|
|
export function getBuiltinLeadToolDefs(): ToolDef[] {
|
|
return getAllToolDefs().filter((def) => def.name !== 'generate_design');
|
|
}
|
|
|
|
/** All tool definitions — canonical schema source for both lead and member registries. */
|
|
export function getAllToolDefs(): ToolDef[] {
|
|
return [
|
|
...getDesignToolDefs(),
|
|
{
|
|
name: 'plan_layout',
|
|
description:
|
|
'Create a root design frame and return a section plan. Use this FIRST before generating content. Returns section names and the root frame ID. Call it again only when you intentionally want a new root frame/artboard.',
|
|
level: TOOL_AUTH_MAP.plan_layout,
|
|
parameters: {
|
|
type: 'object',
|
|
properties: {
|
|
prompt: { type: 'string', description: 'Design description to plan layout for' },
|
|
newRoot: {
|
|
type: 'boolean',
|
|
description:
|
|
'Set true only when you intentionally want to create another root frame/artboard in the same session',
|
|
},
|
|
},
|
|
required: ['prompt'],
|
|
},
|
|
},
|
|
{
|
|
name: 'batch_insert',
|
|
description:
|
|
'Insert PenNode objects into the canvas. MAX 9 nodes per call — call multiple times for more nodes. Each node needs id, type, name. Use _parent field to specify parent node ID.',
|
|
level: TOOL_AUTH_MAP.batch_insert,
|
|
parameters: {
|
|
type: 'object',
|
|
properties: {
|
|
parentId: {
|
|
type: ['string', 'null'],
|
|
description: 'Parent frame ID to insert into (from plan_layout result)',
|
|
},
|
|
nodes: {
|
|
type: 'array',
|
|
items: { type: 'object' },
|
|
description: 'Array of PenNode objects to insert',
|
|
},
|
|
},
|
|
required: ['parentId', 'nodes'],
|
|
},
|
|
},
|
|
{
|
|
name: 'insert_node',
|
|
description:
|
|
'Insert a new node into the document tree. Always call snapshot_layout or batch_get first. ' +
|
|
'Use "after" to insert next to a sibling (auto-finds parent and position), or "parent" for explicit placement.',
|
|
level: TOOL_AUTH_MAP.insert_node,
|
|
parameters: {
|
|
type: 'object',
|
|
properties: {
|
|
after: {
|
|
type: 'string',
|
|
description:
|
|
'Insert after this sibling node ID (preferred). Automatically uses the same parent and places the new node right after it.',
|
|
},
|
|
parent: {
|
|
type: ['string', 'null'],
|
|
description:
|
|
'Explicit parent node ID. Use "after" instead when adding next to existing elements.',
|
|
},
|
|
data: {
|
|
type: 'object',
|
|
description: 'PenNode data (type, name, width, height, fills, children, etc.)',
|
|
},
|
|
pageId: {
|
|
type: 'string',
|
|
description: 'Target page ID (optional, defaults to active page)',
|
|
},
|
|
},
|
|
required: ['data'],
|
|
},
|
|
},
|
|
{
|
|
name: 'find_empty_space',
|
|
description: 'Find empty space on the canvas for placing new content',
|
|
level: TOOL_AUTH_MAP.find_empty_space,
|
|
parameters: {
|
|
type: 'object',
|
|
properties: {
|
|
width: { type: 'number', description: 'Required width' },
|
|
height: { type: 'number', description: 'Required height' },
|
|
pageId: { type: 'string', description: 'Target page ID (optional)' },
|
|
},
|
|
required: ['width', 'height'],
|
|
},
|
|
},
|
|
{
|
|
name: 'get_selection',
|
|
description: 'Get the currently selected nodes on the canvas with their full data',
|
|
level: TOOL_AUTH_MAP.get_selection,
|
|
parameters: {
|
|
type: 'object',
|
|
properties: {},
|
|
},
|
|
},
|
|
];
|
|
}
|
|
|
|
export { TOOL_AUTH_MAP };
|