1 commit

Author	SHA1	Message	Date
lefarcen	204599a7ae	feat(analytics): ship PostHog v2 event schema (#2285) ... * feat(analytics): ship PostHog v2 event schema Aligns the PostHog wire format with the product team's v2 tracking spec (Open Design 埋点文档 2.0). The previous v1 catalogue defined a flat per-page event name (home_view / studio_click / settings_view…); v2 collapses everything to four core events identified through the page_name + area + element triplet so dashboards can group by surface without owning a separate event per page. Key changes - packages/contracts/src/analytics: collapse to page_view / ui_click / surface_view / *_result event names; bump EVENT_SCHEMA_VERSION to 2; rename the wire field anonymous_id → device_id (value unchanged); promote the configure-state triplet (has_available_configure_cli / configure_type / configure_availability) to a global PostHog register so every event inherits it without per-helper boilerplate. - apps/web/src/analytics: rewrite the 43 trackXxx helpers behind the new typed catalogue; opt out of PostHog's built-in UA bot filter so legitimate embedded webviews, fingerprinted browsers, and the Playwright-based e2e runs ingest captures (the Privacy → "Share usage data" toggle remains the single consent gate). - apps/web components: wire P0/P1/P2 click + view + result surfaces end-to-end — left nav, toolbar, home chat composer, recent projects, new project modal, plugins / design systems / integrations / automations pages, file manager, artifact toolbar/header/share popup, feedback panel, settings sidebar / language / appearance / notifications / pets / privacy / connectors. Fixes the v1 feedback bug where action=clear_feedback_rating shipped rating=null instead of the rating being cleared. - apps/daemon: extend run_created / run_finished with the v2 context (entry_from / project_kind / target_platforms / fidelity / connectors / etc.), add explicit error_code classification on result=failed (run.errorCode → AGENT_SIGNAL_* → AGENT_EXIT_* → AGENT_TERMINATED_UNKNOWN), and read device_id from the new x-od-analytics-device-id header. Also moves the run_created / run_finished emission to the canonical /api/runs handler in server.ts; the chat-routes copy was shadowed by Express's earlier registration and never executed, which also meant run.clientType never made it to Langfuse — fixed in the same move. Verification - pnpm guard / pnpm typecheck clean for daemon, web, and contracts. - pnpm --filter @open-design/web test: 1645/1645 passing. - End-to-end smoke through Playwright + local PostHog ingest project 420348: every page_view (home/projects/automations/design_systems/ plugins/integrations/chat_panel/file_manager), every nav element, the new_project_modal surface_view + tab + create flow, the plugin_replacement_modal surface_view, settings_view across nine sections, settings_cli_test_result (codex CLI), the project_create_result success path, and run_created + run_finished (result=failed, error_code=AGENT_EXIT_1) all reached PostHog with the v2 schema and the expected device_id / page_name / area / element / fidelity / target_platforms props. The remaining *_result events (artifact_export / feedback_submit / file_upload / plugin_replacement / settings_byok_test / settings_connector_auth) are wired in code; production traffic will trigger them. * fix(analytics): preserve style category on design-systems surface chip switch The merge resolution in DesignSystemsTab incorrectly re-introduced a `setCategory('All')` call alongside the new `trackDesignSystemsTopClick` emit. main intentionally keeps the active style category when the surface filter refines within it; the regression was caught by the existing "keeps the style category when a surface chip refines within it" test in tests/components/DesignSystemsTab.test.tsx. * fix(analytics): address review — senseaudio passthrough + daemon-side configure-state Two follow-ups from the v2 schema review on #2285: 1. `byokProtocolToTracking()` was still falling through to `null` for `senseaudio` even though the v2 BYOK provider enum now lists it. Every `SettingsDialog` BYOK call site guards on `if (byokProviderId)`, so a user on SenseAudio was silently dropping the provider-option, field-focus, and test-result captures. Added the missing case so SenseAudio gets the same analytics coverage as the other providers. 2. The daemon-authoritative `run_created` / `run_finished` events were missing the configure-state triplet (`has_available_configure_cli` / `configure_type` / `configure_availability`) that v2 promotes to a global register on the web side. Daemon captures don't go through the PostHog global register, so dashboards couldn't segment run lifecycle by execution setup after the migration. The fix derives the triplet server-side from `detectAgents()` and the request's `agentId` before `design.analytics.capture(...)`: - has_available_configure_cli: any CLI on PATH reports installed - configure_type: 'local_cli' when the run targets an installed CLI, otherwise 'unknown' (daemon can't see BYOK keys, which live in web-client storage) - configure_availability: 'available' / 'unavailable' / 'unknown' based on the requested agent's install status, with a fallback to 'available' when any CLI is installed This keeps the v2 schema consistent across both daemon-side and web-side captures. * fix(analytics): wire setConfigureGlobals so browser events carry fresh state Third follow-up from the v2 schema review on #2285. The previous fix addressed senseaudio + daemon-side configure-state, but reviewer flagged that `setConfigureGlobals` was still defined-only — no caller — so every browser-side capture inherited the boot defaults (`has_available_configure_cli=false`, `configure_type='unknown'`, `configure_availability='unknown'`). PostHog dashboards therefore could not segment the new `page_view` / `ui_click` / `surface_view` events by execution setup after a user configured their environment. Changes: - `packages/contracts/src/analytics/events.ts` — add a pure `deriveConfigureGlobals(mode, agentId, agents, byokConfigured)` helper so the web client and the daemon can derive the triplet from the same source of truth. The helper covers all 5 `configure_type` buckets (`local_cli` / `byok` / `both` / `none` / `unknown`) and the 3 `configure_availability` buckets (`available` / `unavailable` / `unknown`). - `apps/web/src/App.tsx` — add a useEffect that re-derives the triplet whenever the user changes execution mode, selects a new CLI, saves a BYOK key, or the detected-agent list refreshes, then pushes it to PostHog via `analytics.setConfigureGlobals(...)`. The setter goes through the provider so the analytics module stays the single source of truth. - `apps/web/src/analytics/provider.tsx` — expose `setConfigureGlobals` on the analytics context and the test stub so consumers route through the provider boundary. - `apps/daemon/src/server.ts` — switch the daemon-side derive in `/api/runs` to the shared `deriveConfigureGlobals` helper so the authoritative run_created/run_finished captures match the web-side payload. BYOK credentials live in the web client and stay invisible to the daemon, so the daemon arm passes `byokConfigured: undefined` and falls back to the installed-CLI signal. - `apps/web/tests/analytics-configure-globals.test.ts` — new regression test that pins the derive behavior across all branches and confirms the setter actually mutates the client-side store. Locks the wire-up so a future refactor can't silently turn the setter back into a no-op. Verification: pnpm guard clean; daemon / web typecheck clean; web tests 1703/1703 passing (up from 1696 — 7 new tests in the configure-globals suite). * fix(analytics): emit projects page_view + drop misattributed chat_panel source Fourth review pass on PR #2285. Two follow-ups from mrcfps: 1. DesignsTab (projects landing) was emitting click events but no matching page_view. Opening /projects without clicking anything left the surface invisible in PostHog. Added a once-per-mount trackPageView({ page_name: 'projects' }) with the same ref-keyed pattern HomeView / PluginsView use. 2. ChatComposer was hard-coding source: 'recent_project' on every chat_panel page_view. The web router currently only carries projectId / conversationId / fileName, so we cannot distinguish a New-project launch from a template-pick or a Recent-projects click from this layer. A false constant would over-attribute every chat launch to 'recent_project' and break the funnel slice this schema was meant to unlock. Dropped the field for now — better no source than the wrong source — until the router grows a launch-source channel; the field is still defined as optional on PageViewProps so the channel can land in a follow-up PR. Verification: web typecheck clean; web tests 1703/1703 passing. * fix(analytics): correct plugin-replacement async result + heterogeneous upload + missing requestId Three follow-ups from the fifth review pass on PR #2285: 1. **plugin_replacement_result emitted before the apply settled** (`apps/web/src/components/HomeView.tsx`). The modal's confirm action was a synchronous wrapper around an async `usePlugin(...)` call, so the surrounding try/catch never observed real failures and every attempt was reported as `result=success`. Changed `PendingReplacement. confirm` to return `Promise<void>`, made the wrapper return the underlying promise, and moved the analytics emit into an async IIFE in the click handler so the success/failure branches reflect the actual outcome. 2. **file_upload_result mis-typed heterogeneous batches** (`apps/web/src/components/FileWorkspace.tsx`). The earlier implementation only inspected `picked[0]`, so a mixed batch like `image.png + demo.mp4` reported `file_type=image`. Per the comment above the block ("mixed batches collapse to other"), the implementation now maps every file to a tracking type, collapses to `other` when more than one distinct type is present, and falls back to the single type otherwise. 3. **project_create_result lost the click→result correlation id** (`apps/web/src/components/NewProjectPanel.tsx`). The click event no longer carried the locally-generated `requestId` that `project_create_result` keeps, so the two could not be joined. `trackNewProjectModalElementClick()` now accepts an optional `{ requestId }`, mirroring the other helpers, and the create-button click threads the same id used for the result. Verification: web typecheck clean; web tests 1703/1703 passing. * fix(analytics): gate configure-state on agents probe + drop unsent run_created fields Two follow-ups from the sixth review pass on PR #2285: 1. **Cold-start configure-state was stamped before fetchAgents() landed** (`apps/web/src/App.tsx`). The useEffect that pushes the v2 triplet into the PostHog global register fired on first paint with `agents=[]`, so the first home/projects/plugins page_view reported `has_available_configure_cli=false` / `configure_availability= unavailable` even on machines that did have an installed CLI. The effect now waits on `agentsLoading === false` and leaves the boot defaults ('unknown'/'unknown') in place until the probe resolves. 2. **Daemon read run-context fields the web never sends** (`apps/daemon/src/server.ts`). The daemon-side run_created / run_finished baseProps read `projectKind`, `entryFrom`, `projectSource`, `targetPlatforms`, `companionSurfaces`, `fidelity`, `connectors`, `useSpeakerNotes`, `includeAnimations`, `referenceTemplate`, and `aspect` from `req.body`, but `packages/contracts/src/api/chat.ts` and `apps/web/src/providers/daemon.ts` don't carry those keys on the wire. Reading them therefore always produced null/undefined. Dropped the unsent fields from the daemon capture; a follow-up can extend the create payload to thread the real context through. The `design_system_id` field stays because the chat contract does send it. Tests: added 3 regression tests in `tests/analytics-configure-globals. test.ts` covering the boot-time gating contract (empty agents + daemon mode → unavailable / local_cli; installed agent → available; undefined agents list → unavailable). Verification: web typecheck clean; daemon typecheck clean; web tests 1706/1706 passing (up from 1703 — 3 new cold-start tests). * fix(analytics): pin mode='daemon' so missing-agent run reports unavailable Eleventh review pass on PR #2285. mrcfps flagged that `apps/daemon/src/server.ts` was calling `deriveConfigureGlobals(...)` without `mode`, so the helper fell through to the generic branch. Result: a run for an uninstalled agent was tagged `configure_availability: 'available'` whenever any OTHER CLI was on PATH, because the generic branch only looks at the cohort-wide "any installed?" signal. That precisely undermines the slice the daemon emit is trying to power. The daemon's /api/runs handler is always a daemon-mode capture (daemon is the local CLI runner — BYOK lives in the web layer), so we now pin `mode: 'daemon'` on the call site. The helper then judges `configure_availability` from the REQUESTED agent's install status and reports `unavailable` when the user picked an agent that is not installed, even if peers are. Added a regression case in `tests/analytics-configure-globals.test.ts`: `{ mode: 'daemon', agentId: 'codex', agents: [{claude,true},{codex,false}] }` → `{ has_available_configure_cli: true, configure_type: 'local_cli', configure_availability: 'unavailable' }`. Verification: daemon typecheck clean; web tests 1707/1707 passing (up from 1706 — 1 new regression test). * fix(analytics): hoist chat_panel page_view + thread requestId - Move chat_panel page_view emit from ChatComposer to ProjectView so it survives activeConversationId-driven ChatPane remounts. ProjectView keys the dedupe ref by project.id; the composer drops its duplicate. - Thread { requestId } into trackAssistantFeedbackReasonSubmitClick so the click pairs with the existing feedback_submit_result on the same request id (mirrors the trackNewProjectModalElementClick pattern). * fix(analytics): keep v2 super-props alive across reset and stamp design_system_source - Snapshot the register payload in client.ts on PostHog init and re-register it from applyConsent(true) and applyIdentity() so a privacy-toggle or Delete-my-data rotation does not resume capture without event_schema_version / device_id / session_id / locale / configure-state globals. setConfigureGlobals() also patches the cache so a later restore picks up the current configure state. - Stamp design_system_source on daemon-side run_created / run_finished (it is required by RunCreatedProps / RunFinishedProps). Daemon can't tell default vs user_selected vs inherited from the wire, so it derives 'unknown' when designSystemId is present, 'not_applicable' otherwise — a follow-up that threads designSystemSource through CreateRunRequest can replace this with the precise source.	2026-05-20 13:04:20 +08:00