mirror of
https://github.com/nexu-io/open-design.git
synced 2026-05-31 19:04:39 +07:00
e1bc83a476
* feat(analytics): scaffold PostHog product-analytics integration
- Add @open-design/contracts/analytics subpath with the 17 P0 event
payload types, header constants, and code↔CSV enum mapping helpers.
- Add apps/daemon/src/analytics.ts with env-gated posthog-node client,
request-scoped analytics context reader, and artifact-id anonymizer.
- Expose GET /api/analytics/config so the web bundle never embeds the
PostHog key at build time; daemon owns POSTHOG_KEY / POSTHOG_HOST.
- Add apps/web/src/analytics module (identity + lazy posthog-js client
+ React provider) and mount it under <I18nProvider> in app/layout.
No event wiring yet — that lands in the next commit alongside trigger
points (App.tsx, EntryView, NewProjectPanel, SettingsDialog, FileViewer,
runs.ts).
* feat(analytics): wire app_launch, home_view, home_click, project_create_result
- App.tsx: fire app_launch once after first effect tick. handleCreateProject
now emits project_create_result on both success and failure paths.
- EntryView.tsx: home_view (page) gated on agents loading so
has_available_cli isn't transiently false; home_view (asset_panel) fires
per top-tab change with the right result_count.
- NewProjectPanel.tsx: home_click create_button fires before delegating to
the parent; a fresh request_id is generated here and threaded through
onCreate so the matching project_create_result stitches via $insert_id.
- contracts/analytics: tighten createTabToTracking and topTabToTracking
for the worktree branch's renamed tabs (live-artifact, templates).
* feat(analytics): wire settings_view + 3 settings_click events
- settings_view fires on dialog mount and on every section switch,
carrying the active section (mapped via settingsSectionToTracking
for the 16-section worktree layout), execution_mode, and the
selected CLI provider id when present.
- settings_click execution_mode_tab: setMode now emits before/after
values whenever the user toggles between Local CLI and BYOK.
- settings_click cli_provider_card: agent card onClick reports
cli_provider_id via agentIdToTracking (kiro → other).
- settings_click byok_field: onFocus added to api_key, model select,
and base_url inputs; provider_id widened to include google so the
worktree's Gemini protocol slot type-checks.
* feat(analytics): wire studio_view + studio_click chat, studio_view artifact
- packages/contracts/src/analytics/artifact-id.ts: FNV-1a 64-bit helper
produces a 16-hex anonymized id for (projectId, fileName). Stable
cross-platform so the daemon and the web bundle resolve the same id
without a Web Crypto round-trip; daemon now re-exports it.
- ChatComposer: studio_view chat_panel fires once per project mount,
studio_click chat_composer fires on attachment + send buttons with
estimated user_query_tokens (length/4) and has_attachment.
- FileViewer: studio_view artifact fires once per (project, file) at
the dispatcher level, before any sub-viewer renders, with
artifact_kind derived from the renderer registry / file.kind table.
- Widen TrackingExportFormat to include markdown and cloudflare_pages
so the worktree branch's full share menu can emit verbatim.
* feat(analytics): wire studio_click share_option + artifact_export_result
HtmlViewer's share menu now emits both events per click via a
fireShareExport helper:
- studio_click share_option fires immediately on click with the chosen
export_format and a fresh request_id.
- artifact_export_result fires when the export resolves — success for
sync exporters (html, markdown, template) the moment the call
returns, success/failed for async exporters (pdf, zip, deploy)
via .then/.catch. The same request_id threads both events so
PostHog stitches click → result via $insert_id.
DEPLOY_PROVIDER_OPTIONS maps to the CSV's vercel / cloudflare_pages
slots; markdown is now a first-class export_format value.
Also ignore .env.local so local POSTHOG_KEY / .env-style secrets
don't get committed.
* feat(analytics): emit run_created and run_finished from the daemon
POST /api/runs now reads the analytics context off the
x-od-analytics-* headers the web client sets on every fetch, then:
- Captures run_created with project_id, conversation_id, run_id,
model_id, agent_provider_id (mapped via agentIdToTracking),
skill_id, design_system_id, plus the token_count_source marker.
- Schedules a run_finished capture on runs.wait(run) resolution,
mapping succeeded/canceled/failed to success/cancelled/failed and
reporting total_duration_ms.
Both events use a stable insert_id derived from the same uuid so
PostHog dedupes the daemon-side mirror against any future
web-side capture without double-counting.
Token sub-fields (user_query_tokens/system_prompt_tokens/...) stay
omitted in v1 — the claude-stream parser only exposes input/output
totals today. See tracking-doc-issues.md §3.2.
* feat(analytics): emit settings_cli_test_result + settings_byok_test_result
The original BLOCKING-list assumed these CSV P0 events were not
implementable in this branch because main lacked Test buttons. The
worktree HEAD actually wires `handleTestAgent` and `handleTestProvider`
in SettingsDialog, so both events are now in scope.
- handleTestAgent emits settings_cli_test_result on success and
failure paths with cli_provider_id mapped via agentIdToTracking,
result drawn from result.ok / catch branch, error_code from
result.kind or the thrown error name, and duration_ms timed via
performance.now().
- handleTestProvider emits settings_byok_test_result analogously,
using apiProtocol (anthropic|openai|azure|ollama|google) directly
as provider_id — wider than the CSV's 5-value enum, documented in
tracking-doc-issues.md §2.5.
Contracts: add SettingsCliTestResultProps / SettingsByokTestResultProps
plus matching track* helpers. AnalyticsEventName union now covers all
14 P0 events this branch supports.
* feat(analytics): gate PostHog on the existing telemetry.metrics consent
The integration now reuses the same first-launch privacy banner +
Settings → Privacy toggle that gates Langfuse, so a single user
decision controls both telemetry sinks.
- /api/analytics/config now consults the persisted AppConfigPrefs:
it returns enabled=true only when POSTHOG_KEY is set AND the user
has chosen "Share usage data" (telemetry.metrics === true). The
response also echoes installationId so the web client uses the
same anonymous id Langfuse keys off of — one identity per install,
shared across both sinks.
- Web AnalyticsProvider:
- Bootstrap fetch resolves installationId and threads it through
the x-od-analytics-anonymous-id header on every /api/* fetch,
so daemon-side captures (run_created / run_finished /
project_create_result) land on the same person record.
- Exposes a setConsent(granted) method that calls posthog-js's
opt_in_capturing / opt_out_capturing, wired from App.tsx via a
useEffect watching config.telemetry?.metrics. Toggling Privacy
→ metrics now stops/resumes events immediately, no reload.
- app_launch additionally gates on telemetry.metrics so a freshly-
declined user fires nothing, and a freshly-opted-in user fires on
the next reload.
* feat(packaging): bake POSTHOG_KEY into packaged daemon spawn env
Wires PostHog product analytics through the same Langfuse-style build-
secret pipeline so official Open Design builds ship with the key while
fork builds compile without it (the integration short-circuits cleanly
when POSTHOG_KEY is absent).
tools/pack
- resolveToolPackConfig reads POSTHOG_KEY / POSTHOG_HOST from
process.env at packaging time, validates them (no whitespace in the
key, http(s) URL for host, trailing-slash strip), and stamps them on
ToolPackConfig. Fork builds without the env vars simply omit the
fields; the daemon-side gate keeps things off in that case.
- Mac, Windows, and Linux packaged-config writers each append the two
fields to open-design-config.json next to the existing
telemetryRelayUrl entry.
apps/packaged
- RawPackagedConfig / PackagedConfig surface posthogKey / posthogHost
so the Electron entry and headless entry both forward them to the
daemon sidecar.
- buildPackagedDaemonSpawnEnv emits POSTHOG_KEY / POSTHOG_HOST into
the daemon child env when present. The daemon's existing analytics
module reads these via process.env — no daemon-side changes needed.
- The headless packaged path falls back to process.env for fields the
builder hasn't injected, mirroring how OPEN_DESIGN_TELEMETRY_RELAY_URL
is read there.
CI
- release-beta.yml and release-stable.yml expose POSTHOG_KEY (secret)
and POSTHOG_HOST (var) at workflow-env scope so every packaging job
inherits them. PR / fork builds without these set simply skip the
bake step.
Tests
- tools/pack: config.test.ts covers bake-through, fork-build omission,
whitespace rejection, invalid-URL rejection, and trailing-slash
normalization.
- apps/packaged: sidecars.test.ts covers buildPackagedDaemonSpawnEnv
forwarding the keys when present and omitting them when null.
* feat(analytics): enable PostHog autocapture + perf + exceptions
Flip on the PostHog SDK's automatic diagnostic features so we capture
click paths, page transitions, web vitals, dead clicks, and browser
exceptions without scattering instrumentation through the codebase.
Privacy defense lives in one place — apps/web/src/analytics/scrub.ts —
wired in via posthog-js's `before_send` hook so every outgoing event
passes through the same audit point:
- $autocapture / $rageclick / $dead_click / $copy_autocapture:
strips $el_text and value/placeholder/aria-label attrs from any
input, textarea, password input, or contenteditable element. PostHog
autocapture does not capture input.value by default, but $el_text
on a <textarea> reflects the typed content — that's the prompt
body for us, so it has to be scrubbed every time.
- $pageview / $pageleave: drops query string and fragment from
$current_url / $referrer so any future ?q=… can't leak.
- $exception: rewrites file:// and absolute filesystem paths in
stack frames to app://apps/<repo-relative> so we don't ship the
user's home directory.
- Suppresses $opt_in entirely — duplicate of our explicit
setConsent toggle in App.tsx.
Element-level defense in depth is limited to the single most sensitive
surface: the chat composer textarea gets `ph-no-capture` so PostHog
never even generates an event for clicks inside that subtree. Every
other input relies on scrub.ts — sprinkling the class through every
form would be noisy and easy to forget on new surfaces.
The existing Privacy → "Share usage data" toggle continues to gate
every new feature: posthog-js's opt_out_capturing() halts autocapture,
$pageview, $exception, web vitals, and dead clicks alongside the
explicit capture() calls — one global switch.
11 unit tests pin the scrub rules in apps/web/tests/analytics-scrub.test.ts.
* ci(nix): bump pnpmDepsHash for posthog-js + posthog-node additions
Adding posthog-js to apps/web and posthog-node to apps/daemon changed
pnpm-lock.yaml, which Nix's fixed-output pnpmDeps derivation pins by
sha256. The CI nix flake check failed with:
specified: sha256-KF3Mld72/iau+pJmA7HvnanRx8VLtDP0N624SKrtrrc=
got: sha256-PGFgX4lYyeH2TRAXfUq52A3EOa6bb1gO59hPsXhEk3s=
Copy the new hash into both nix/package-web.nix and
nix/package-daemon.nix per the procedure documented in nix/README.md
§"First-build hash pinning".
* feat(analytics): unify PostHog identity with Langfuse installationId
PostHog's distinct_id is the installationId stamped by /api/analytics/
config; Langfuse already reads the same id off app-config.json to
populate trace.userId. With both sinks keying off the same anonymous
identity, dashboards can correlate user actions (PostHog events) with
LLM runs (Langfuse traces) without re-identifying.
Two gaps closed:
1. applyConsent(false) — clear posthog-js's persisted ph_*_posthog
localStorage entry on opt-out via posthog.reset(). Without this, a
user who opts out, then clicks Delete my data, then re-opts in
would see PostHog stitch their new session to the deleted identity
because bootstrap.distinctID only takes effect on first init.
2. applyIdentity(newInstallationId) — Delete my data rotates the
installationId in app-config; App.tsx now watches config.installationId
and calls posthog.reset() then identify(newId) so the next event
batch is fully decoupled from the deleted one. Idempotent on
same-id re-renders so benign config refreshes don't churn PostHog
identities.
The fetch wrapper's x-od-analytics-anonymous-id header also flips to
the new id on rotation so daemon-side captures (run_created /
run_finished) land on the same person record from the very next API
call, not after a reload.
The end-to-end rotation flow is verified against a live PostHog
project; these unit tests pin the safety guards (no-client paths, null
inputs) since stubbing posthog-js's init-loaded callback chain is
brittle.
* fix(langfuse): require both metrics AND content consent for trace reports
Tightens the Langfuse gate so a user who shares anonymous metrics but
NOT conversation content stops emitting Langfuse traces entirely —
Langfuse is used for turn-quality evals which only make sense with
prompt/output bodies. PostHog (product analytics, content-free) stays
gated on `metrics` alone and is unaffected.
i18n: "Conversation content" → "Conversation and tool content" with
hints expanded to mention tool inputs/outputs so the consent surface
matches what the trace actually carries (en + zh-CN).
Bundled here per PR scope — change originated outside this PostHog
PR but lands cleanly on the same files; gating Langfuse strictly
on `content` makes the dual-sink consent model (PostHog = metrics,
Langfuse = metrics + content) symmetric across both i18n locales and
the daemon-side gate.
* feat(analytics): wire byok_provider_option + fix PR review P1s
Adds the BYOK protocol-chip click event (5-value provider_id mirroring
the apiProtocol Settings UI) and resolves four P1 review threads on
PR #1428.
byok_provider_option:
- New SettingsClickByokProviderOptionProps in contracts (provider_id =
anthropic|openai|azure|google|ollama; maps to CSV's 5 values per
tracking-doc-issues.md §2.5).
- trackSettingsClickByokProviderOption helper in apps/web/src/analytics.
- SettingsDialog hooks it on the protocol-chip onClick alongside the
existing setApiProtocol call; is_selected reflects whether the chip
was already active.
Review fixes:
1. client.ts (Siri-Ray): clear `initPromise` when the resolution is
null so a Privacy → metrics opt-in after a previous decline triggers
a fresh /api/analytics/config fetch. Without this, the disabled
response was cached forever — first-session opt-in needed a reload
to start sending PostHog events.
2. provider.tsx (Siri-Ray): replace `url.includes('/api/')` with a
strict same-origin + /api/ pathname check (shared
`isSameOriginApiCall` helper). Outbound third-party URLs containing
`/api/` (e.g. provider.example.com/api/x) no longer receive our
x-od-analytics-* headers.
3. provider.tsx (codex-connector, lefarcen): gate header injection on
`resolvedAnonId` being non-null. When Privacy → metrics is off,
/api/analytics/config returns enabled=false → resolvedAnonId stays
null → wrapper never installs → daemon can't read consent-bearing
headers → no daemon-side PostHog event. setConsent now also clears
resolvedAnonId on opt-out and re-fetches on opt-in.
4. daemon/analytics.ts (defense in depth): createAnalyticsService now
takes dataDir and capture() re-reads app-config to check
telemetry.metrics inside the fire-and-forget wrapper. Even if a
stale header somehow reaches the daemon after opt-out, the capture
is dropped before posthog-node.capture is called.
* fix(web): place "Share usage data" on the right in privacy consent banner
Swap button order in PrivacyConsentModal and the in-settings ConsentCard
so the affirmative "Share usage data" lands on the right and "Not now"
on the left. Matches the OK-on-the-right pattern users expect for
primary actions.
Both buttons keep equal visual prominence (same .privacy-consent-action
styling) so the swap doesn't change the EDPB equal-prominence stance
called out in the original Langfuse telemetry spec.
* feat(analytics): populate run_finished token totals from claude-stream usage
Daemon's claude-stream parser already emits agent usage events with
input_tokens / output_tokens totals; the run service buffers them in
run.events and Langfuse reads them out the same way. The run_finished
PostHog event was leaving these fields empty.
Scan run.events for the most recent agent usage frame on terminal
transition and emit input_tokens / output_tokens / total_tokens when
present. token_count_source flips to 'provider_usage' only when at
least one count landed; runs without provider-side usage data keep
'unknown'.
Provider does not break the input down into the 7 sub-fields the
tracking doc lists (memory / context / attachment / system_prompt /
…); those stay omitted until a parser change exposes them.
* feat(analytics): estimate user_query_tokens from prompt length
The user_query_tokens field for run_created / run_finished was hardcoded
to 0. We can't tokenize without bundling a model-specific tokenizer, but
the character/4 heuristic is the industry-standard estimate when one
isn't available and is enough for funnel analysis (prompt-length cohorts,
short-vs-long-query conversion rates).
Extracted from req.body via the same telemetryPromptFromRunRequest
pattern the daemon already uses for langfuse-bridge (currentPrompt then
message fallback). Only the integer count goes to PostHog — the prompt
text itself never leaves the daemon.
token_count_source flips appropriately:
- run_created with a prompt: 'estimated' (was 'unknown')
- run_created with no prompt: 'unknown'
- run_finished with provider usage: 'provider_usage' (overrides
baseProps' 'estimated' value)
- run_finished without provider usage: inherits 'estimated' or 'unknown'
from baseProps so input/output absent doesn't mask the estimate.
59 lines
1.6 KiB
JSON
59 lines
1.6 KiB
JSON
{
|
|
"name": "@open-design/contracts",
|
|
"version": "0.7.0",
|
|
"private": true,
|
|
"type": "module",
|
|
"description": "Shared pure TypeScript contracts for the Open Design web/daemon boundary.",
|
|
"main": "./dist/index.mjs",
|
|
"types": "./dist/index.d.ts",
|
|
"files": [
|
|
"dist"
|
|
],
|
|
"exports": {
|
|
".": {
|
|
"types": "./dist/index.d.ts",
|
|
"default": "./dist/index.mjs"
|
|
},
|
|
"./api/connectionTest": {
|
|
"types": "./dist/api/connectionTest.d.ts",
|
|
"default": "./dist/api/connectionTest.mjs"
|
|
},
|
|
"./api/orbit": {
|
|
"types": "./dist/api/orbit.d.ts",
|
|
"default": "./dist/api/orbit.mjs"
|
|
},
|
|
"./api/finalize": {
|
|
"types": "./dist/api/finalize.d.ts",
|
|
"default": "./dist/api/finalize.mjs"
|
|
},
|
|
"./api/providerModels": {
|
|
"types": "./dist/api/providerModels.d.ts",
|
|
"default": "./dist/api/providerModels.mjs"
|
|
},
|
|
"./api/research": {
|
|
"types": "./dist/api/research.d.ts",
|
|
"default": "./dist/api/research.mjs"
|
|
},
|
|
"./critique": {
|
|
"types": "./dist/critique.d.ts",
|
|
"default": "./dist/critique.mjs"
|
|
},
|
|
"./analytics": {
|
|
"types": "./dist/analytics/index.d.ts",
|
|
"default": "./dist/analytics/index.mjs"
|
|
}
|
|
},
|
|
"scripts": {
|
|
"build": "node ./esbuild.config.mjs && tsc -p tsconfig.json --emitDeclarationOnly",
|
|
"test": "vitest run",
|
|
"typecheck": "tsc -p tsconfig.json --noEmit && tsc -p tsconfig.tests.json --noEmit"
|
|
},
|
|
"dependencies": {
|
|
"zod": "^3.23.8"
|
|
},
|
|
"devDependencies": {
|
|
"esbuild": "0.27.7",
|
|
"typescript": "^5.6.3",
|
|
"vitest": "^2.1.8"
|
|
}
|
|
}
|