mirror of
https://github.com/nexu-io/open-design.git
synced 2026-06-01 03:14:35 +07:00
3 commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
lefarcen
|
e1bc83a476
|
feat(analytics): PostHog product analytics (P0 events, consent-gated, packaged) (#1428)
* feat(analytics): scaffold PostHog product-analytics integration
- Add @open-design/contracts/analytics subpath with the 17 P0 event
payload types, header constants, and code↔CSV enum mapping helpers.
- Add apps/daemon/src/analytics.ts with env-gated posthog-node client,
request-scoped analytics context reader, and artifact-id anonymizer.
- Expose GET /api/analytics/config so the web bundle never embeds the
PostHog key at build time; daemon owns POSTHOG_KEY / POSTHOG_HOST.
- Add apps/web/src/analytics module (identity + lazy posthog-js client
+ React provider) and mount it under <I18nProvider> in app/layout.
No event wiring yet — that lands in the next commit alongside trigger
points (App.tsx, EntryView, NewProjectPanel, SettingsDialog, FileViewer,
runs.ts).
* feat(analytics): wire app_launch, home_view, home_click, project_create_result
- App.tsx: fire app_launch once after first effect tick. handleCreateProject
now emits project_create_result on both success and failure paths.
- EntryView.tsx: home_view (page) gated on agents loading so
has_available_cli isn't transiently false; home_view (asset_panel) fires
per top-tab change with the right result_count.
- NewProjectPanel.tsx: home_click create_button fires before delegating to
the parent; a fresh request_id is generated here and threaded through
onCreate so the matching project_create_result stitches via $insert_id.
- contracts/analytics: tighten createTabToTracking and topTabToTracking
for the worktree branch's renamed tabs (live-artifact, templates).
* feat(analytics): wire settings_view + 3 settings_click events
- settings_view fires on dialog mount and on every section switch,
carrying the active section (mapped via settingsSectionToTracking
for the 16-section worktree layout), execution_mode, and the
selected CLI provider id when present.
- settings_click execution_mode_tab: setMode now emits before/after
values whenever the user toggles between Local CLI and BYOK.
- settings_click cli_provider_card: agent card onClick reports
cli_provider_id via agentIdToTracking (kiro → other).
- settings_click byok_field: onFocus added to api_key, model select,
and base_url inputs; provider_id widened to include google so the
worktree's Gemini protocol slot type-checks.
* feat(analytics): wire studio_view + studio_click chat, studio_view artifact
- packages/contracts/src/analytics/artifact-id.ts: FNV-1a 64-bit helper
produces a 16-hex anonymized id for (projectId, fileName). Stable
cross-platform so the daemon and the web bundle resolve the same id
without a Web Crypto round-trip; daemon now re-exports it.
- ChatComposer: studio_view chat_panel fires once per project mount,
studio_click chat_composer fires on attachment + send buttons with
estimated user_query_tokens (length/4) and has_attachment.
- FileViewer: studio_view artifact fires once per (project, file) at
the dispatcher level, before any sub-viewer renders, with
artifact_kind derived from the renderer registry / file.kind table.
- Widen TrackingExportFormat to include markdown and cloudflare_pages
so the worktree branch's full share menu can emit verbatim.
* feat(analytics): wire studio_click share_option + artifact_export_result
HtmlViewer's share menu now emits both events per click via a
fireShareExport helper:
- studio_click share_option fires immediately on click with the chosen
export_format and a fresh request_id.
- artifact_export_result fires when the export resolves — success for
sync exporters (html, markdown, template) the moment the call
returns, success/failed for async exporters (pdf, zip, deploy)
via .then/.catch. The same request_id threads both events so
PostHog stitches click → result via $insert_id.
DEPLOY_PROVIDER_OPTIONS maps to the CSV's vercel / cloudflare_pages
slots; markdown is now a first-class export_format value.
Also ignore .env.local so local POSTHOG_KEY / .env-style secrets
don't get committed.
* feat(analytics): emit run_created and run_finished from the daemon
POST /api/runs now reads the analytics context off the
x-od-analytics-* headers the web client sets on every fetch, then:
- Captures run_created with project_id, conversation_id, run_id,
model_id, agent_provider_id (mapped via agentIdToTracking),
skill_id, design_system_id, plus the token_count_source marker.
- Schedules a run_finished capture on runs.wait(run) resolution,
mapping succeeded/canceled/failed to success/cancelled/failed and
reporting total_duration_ms.
Both events use a stable insert_id derived from the same uuid so
PostHog dedupes the daemon-side mirror against any future
web-side capture without double-counting.
Token sub-fields (user_query_tokens/system_prompt_tokens/...) stay
omitted in v1 — the claude-stream parser only exposes input/output
totals today. See tracking-doc-issues.md §3.2.
* feat(analytics): emit settings_cli_test_result + settings_byok_test_result
The original BLOCKING-list assumed these CSV P0 events were not
implementable in this branch because main lacked Test buttons. The
worktree HEAD actually wires `handleTestAgent` and `handleTestProvider`
in SettingsDialog, so both events are now in scope.
- handleTestAgent emits settings_cli_test_result on success and
failure paths with cli_provider_id mapped via agentIdToTracking,
result drawn from result.ok / catch branch, error_code from
result.kind or the thrown error name, and duration_ms timed via
performance.now().
- handleTestProvider emits settings_byok_test_result analogously,
using apiProtocol (anthropic|openai|azure|ollama|google) directly
as provider_id — wider than the CSV's 5-value enum, documented in
tracking-doc-issues.md §2.5.
Contracts: add SettingsCliTestResultProps / SettingsByokTestResultProps
plus matching track* helpers. AnalyticsEventName union now covers all
14 P0 events this branch supports.
* feat(analytics): gate PostHog on the existing telemetry.metrics consent
The integration now reuses the same first-launch privacy banner +
Settings → Privacy toggle that gates Langfuse, so a single user
decision controls both telemetry sinks.
- /api/analytics/config now consults the persisted AppConfigPrefs:
it returns enabled=true only when POSTHOG_KEY is set AND the user
has chosen "Share usage data" (telemetry.metrics === true). The
response also echoes installationId so the web client uses the
same anonymous id Langfuse keys off of — one identity per install,
shared across both sinks.
- Web AnalyticsProvider:
- Bootstrap fetch resolves installationId and threads it through
the x-od-analytics-anonymous-id header on every /api/* fetch,
so daemon-side captures (run_created / run_finished /
project_create_result) land on the same person record.
- Exposes a setConsent(granted) method that calls posthog-js's
opt_in_capturing / opt_out_capturing, wired from App.tsx via a
useEffect watching config.telemetry?.metrics. Toggling Privacy
→ metrics now stops/resumes events immediately, no reload.
- app_launch additionally gates on telemetry.metrics so a freshly-
declined user fires nothing, and a freshly-opted-in user fires on
the next reload.
* feat(packaging): bake POSTHOG_KEY into packaged daemon spawn env
Wires PostHog product analytics through the same Langfuse-style build-
secret pipeline so official Open Design builds ship with the key while
fork builds compile without it (the integration short-circuits cleanly
when POSTHOG_KEY is absent).
tools/pack
- resolveToolPackConfig reads POSTHOG_KEY / POSTHOG_HOST from
process.env at packaging time, validates them (no whitespace in the
key, http(s) URL for host, trailing-slash strip), and stamps them on
ToolPackConfig. Fork builds without the env vars simply omit the
fields; the daemon-side gate keeps things off in that case.
- Mac, Windows, and Linux packaged-config writers each append the two
fields to open-design-config.json next to the existing
telemetryRelayUrl entry.
apps/packaged
- RawPackagedConfig / PackagedConfig surface posthogKey / posthogHost
so the Electron entry and headless entry both forward them to the
daemon sidecar.
- buildPackagedDaemonSpawnEnv emits POSTHOG_KEY / POSTHOG_HOST into
the daemon child env when present. The daemon's existing analytics
module reads these via process.env — no daemon-side changes needed.
- The headless packaged path falls back to process.env for fields the
builder hasn't injected, mirroring how OPEN_DESIGN_TELEMETRY_RELAY_URL
is read there.
CI
- release-beta.yml and release-stable.yml expose POSTHOG_KEY (secret)
and POSTHOG_HOST (var) at workflow-env scope so every packaging job
inherits them. PR / fork builds without these set simply skip the
bake step.
Tests
- tools/pack: config.test.ts covers bake-through, fork-build omission,
whitespace rejection, invalid-URL rejection, and trailing-slash
normalization.
- apps/packaged: sidecars.test.ts covers buildPackagedDaemonSpawnEnv
forwarding the keys when present and omitting them when null.
* feat(analytics): enable PostHog autocapture + perf + exceptions
Flip on the PostHog SDK's automatic diagnostic features so we capture
click paths, page transitions, web vitals, dead clicks, and browser
exceptions without scattering instrumentation through the codebase.
Privacy defense lives in one place — apps/web/src/analytics/scrub.ts —
wired in via posthog-js's `before_send` hook so every outgoing event
passes through the same audit point:
- $autocapture / $rageclick / $dead_click / $copy_autocapture:
strips $el_text and value/placeholder/aria-label attrs from any
input, textarea, password input, or contenteditable element. PostHog
autocapture does not capture input.value by default, but $el_text
on a <textarea> reflects the typed content — that's the prompt
body for us, so it has to be scrubbed every time.
- $pageview / $pageleave: drops query string and fragment from
$current_url / $referrer so any future ?q=… can't leak.
- $exception: rewrites file:// and absolute filesystem paths in
stack frames to app://apps/<repo-relative> so we don't ship the
user's home directory.
- Suppresses $opt_in entirely — duplicate of our explicit
setConsent toggle in App.tsx.
Element-level defense in depth is limited to the single most sensitive
surface: the chat composer textarea gets `ph-no-capture` so PostHog
never even generates an event for clicks inside that subtree. Every
other input relies on scrub.ts — sprinkling the class through every
form would be noisy and easy to forget on new surfaces.
The existing Privacy → "Share usage data" toggle continues to gate
every new feature: posthog-js's opt_out_capturing() halts autocapture,
$pageview, $exception, web vitals, and dead clicks alongside the
explicit capture() calls — one global switch.
11 unit tests pin the scrub rules in apps/web/tests/analytics-scrub.test.ts.
* ci(nix): bump pnpmDepsHash for posthog-js + posthog-node additions
Adding posthog-js to apps/web and posthog-node to apps/daemon changed
pnpm-lock.yaml, which Nix's fixed-output pnpmDeps derivation pins by
sha256. The CI nix flake check failed with:
specified: sha256-KF3Mld72/iau+pJmA7HvnanRx8VLtDP0N624SKrtrrc=
got: sha256-PGFgX4lYyeH2TRAXfUq52A3EOa6bb1gO59hPsXhEk3s=
Copy the new hash into both nix/package-web.nix and
nix/package-daemon.nix per the procedure documented in nix/README.md
§"First-build hash pinning".
* feat(analytics): unify PostHog identity with Langfuse installationId
PostHog's distinct_id is the installationId stamped by /api/analytics/
config; Langfuse already reads the same id off app-config.json to
populate trace.userId. With both sinks keying off the same anonymous
identity, dashboards can correlate user actions (PostHog events) with
LLM runs (Langfuse traces) without re-identifying.
Two gaps closed:
1. applyConsent(false) — clear posthog-js's persisted ph_*_posthog
localStorage entry on opt-out via posthog.reset(). Without this, a
user who opts out, then clicks Delete my data, then re-opts in
would see PostHog stitch their new session to the deleted identity
because bootstrap.distinctID only takes effect on first init.
2. applyIdentity(newInstallationId) — Delete my data rotates the
installationId in app-config; App.tsx now watches config.installationId
and calls posthog.reset() then identify(newId) so the next event
batch is fully decoupled from the deleted one. Idempotent on
same-id re-renders so benign config refreshes don't churn PostHog
identities.
The fetch wrapper's x-od-analytics-anonymous-id header also flips to
the new id on rotation so daemon-side captures (run_created /
run_finished) land on the same person record from the very next API
call, not after a reload.
The end-to-end rotation flow is verified against a live PostHog
project; these unit tests pin the safety guards (no-client paths, null
inputs) since stubbing posthog-js's init-loaded callback chain is
brittle.
* fix(langfuse): require both metrics AND content consent for trace reports
Tightens the Langfuse gate so a user who shares anonymous metrics but
NOT conversation content stops emitting Langfuse traces entirely —
Langfuse is used for turn-quality evals which only make sense with
prompt/output bodies. PostHog (product analytics, content-free) stays
gated on `metrics` alone and is unaffected.
i18n: "Conversation content" → "Conversation and tool content" with
hints expanded to mention tool inputs/outputs so the consent surface
matches what the trace actually carries (en + zh-CN).
Bundled here per PR scope — change originated outside this PostHog
PR but lands cleanly on the same files; gating Langfuse strictly
on `content` makes the dual-sink consent model (PostHog = metrics,
Langfuse = metrics + content) symmetric across both i18n locales and
the daemon-side gate.
* feat(analytics): wire byok_provider_option + fix PR review P1s
Adds the BYOK protocol-chip click event (5-value provider_id mirroring
the apiProtocol Settings UI) and resolves four P1 review threads on
PR #1428.
byok_provider_option:
- New SettingsClickByokProviderOptionProps in contracts (provider_id =
anthropic|openai|azure|google|ollama; maps to CSV's 5 values per
tracking-doc-issues.md §2.5).
- trackSettingsClickByokProviderOption helper in apps/web/src/analytics.
- SettingsDialog hooks it on the protocol-chip onClick alongside the
existing setApiProtocol call; is_selected reflects whether the chip
was already active.
Review fixes:
1. client.ts (Siri-Ray): clear `initPromise` when the resolution is
null so a Privacy → metrics opt-in after a previous decline triggers
a fresh /api/analytics/config fetch. Without this, the disabled
response was cached forever — first-session opt-in needed a reload
to start sending PostHog events.
2. provider.tsx (Siri-Ray): replace `url.includes('/api/')` with a
strict same-origin + /api/ pathname check (shared
`isSameOriginApiCall` helper). Outbound third-party URLs containing
`/api/` (e.g. provider.example.com/api/x) no longer receive our
x-od-analytics-* headers.
3. provider.tsx (codex-connector, lefarcen): gate header injection on
`resolvedAnonId` being non-null. When Privacy → metrics is off,
/api/analytics/config returns enabled=false → resolvedAnonId stays
null → wrapper never installs → daemon can't read consent-bearing
headers → no daemon-side PostHog event. setConsent now also clears
resolvedAnonId on opt-out and re-fetches on opt-in.
4. daemon/analytics.ts (defense in depth): createAnalyticsService now
takes dataDir and capture() re-reads app-config to check
telemetry.metrics inside the fire-and-forget wrapper. Even if a
stale header somehow reaches the daemon after opt-out, the capture
is dropped before posthog-node.capture is called.
* fix(web): place "Share usage data" on the right in privacy consent banner
Swap button order in PrivacyConsentModal and the in-settings ConsentCard
so the affirmative "Share usage data" lands on the right and "Not now"
on the left. Matches the OK-on-the-right pattern users expect for
primary actions.
Both buttons keep equal visual prominence (same .privacy-consent-action
styling) so the swap doesn't change the EDPB equal-prominence stance
called out in the original Langfuse telemetry spec.
* feat(analytics): populate run_finished token totals from claude-stream usage
Daemon's claude-stream parser already emits agent usage events with
input_tokens / output_tokens totals; the run service buffers them in
run.events and Langfuse reads them out the same way. The run_finished
PostHog event was leaving these fields empty.
Scan run.events for the most recent agent usage frame on terminal
transition and emit input_tokens / output_tokens / total_tokens when
present. token_count_source flips to 'provider_usage' only when at
least one count landed; runs without provider-side usage data keep
'unknown'.
Provider does not break the input down into the 7 sub-fields the
tracking doc lists (memory / context / attachment / system_prompt /
…); those stay omitted until a parser change exposes them.
* feat(analytics): estimate user_query_tokens from prompt length
The user_query_tokens field for run_created / run_finished was hardcoded
to 0. We can't tokenize without bundling a model-specific tokenizer, but
the character/4 heuristic is the industry-standard estimate when one
isn't available and is enough for funnel analysis (prompt-length cohorts,
short-vs-long-query conversion rates).
Extracted from req.body via the same telemetryPromptFromRunRequest
pattern the daemon already uses for langfuse-bridge (currentPrompt then
message fallback). Only the integer count goes to PostHog — the prompt
text itself never leaves the daemon.
token_count_source flips appropriately:
- run_created with a prompt: 'estimated' (was 'unknown')
- run_created with no prompt: 'unknown'
- run_finished with provider usage: 'provider_usage' (overrides
baseProps' 'estimated' value)
- run_finished without provider usage: inherits 'estimated' or 'unknown'
from baseProps so input/output absent doesn't mask the estimate.
|
||
|
lefarcen
|
43f7fc536a
|
Add Langfuse telemetry relay (#1296)
* Add Langfuse telemetry relay * Configure telemetry worker custom domain * Add telemetry relay health check * Harden telemetry relay config |
||
|
lefarcen
|
afb331a288
|
feat: add opt-in Langfuse telemetry (#800)
* docs(specs): add langfuse telemetry change spec
Captures the design for forwarding completed agent runs to Langfuse,
including data-model mapping, field-budget caps, privacy gates,
build-secret injection, GDPR right-to-deletion approach, and the
resolved decisions on default consent, identifier shape, region, and
ownership.
* feat(daemon): add langfuse-trace module and telemetry prefs
Adds the dependency-free building blocks for forwarding completed
agent runs to Langfuse. Two layers:
- AppConfigPrefs gains installationId and a TelemetryPrefs object with
metrics / content / artifactManifest gates. The daemon validator
treats telemetry like agentModels — replace-on-write, drop-when-empty,
reject non-boolean inner values.
- New langfuse-trace.ts builds a {trace-create, generation-create}
pair from a ReportContext, capping prompt at 8 KB, output at 16 KB,
artifacts at 50 entries, and dropping any batch larger than 1 MB
before send. reportRunCompleted is no-op when LANGFUSE_PUBLIC_KEY /
LANGFUSE_SECRET_KEY are unset (so dev runs and forks never emit) and
short-circuits on prefs.metrics === false.
Server-side wiring into the run-close path lands in a follow-up.
* fix(langfuse): default to US Langfuse region
End-to-end smoke against the project's actual dev key on 2026-05-07
returned 401 from cloud.langfuse.com (EU) and 207 from
us.cloud.langfuse.com (US), confirming the org lives in US. Update the
default base URL, the matching test, and the spec's Q3 decision row to
match. Self-hosted or EU-region operators can still override via the
LANGFUSE_BASE_URL env var.
* feat(daemon): wire langfuse trace forwarding into run-close
Adds the daemon-side glue to forward completed agent runs:
- runs.ts gains an optional onTerminate hook fired once per run after it
reaches a terminal state. Errors thrown from the hook are caught and
logged, never propagated, so telemetry can never break the run path.
- New langfuse-bridge.ts assembles a ReportContext from the in-memory
run record, the conversation's persisted assistant message, and the
user's app-config preferences. It tolerates a missing message (e.g.
when web has not yet PUT the final delta) and a missing app-config.
- server.ts stashes the original user prompt on the run object inside
startChatRun so the bridge can include it without crossing the
createChatRunService boundary, and registers the hook callback when
building the run service.
Behavior remains a no-op unless LANGFUSE_PUBLIC_KEY / LANGFUSE_SECRET_KEY
are set in the daemon env AND telemetry.metrics is true in app-config.
A live smoke against us.cloud.langfuse.com on 2026-05-07 confirmed the
matching trace + generation schema is accepted (HTTP 207, both events
201 created).
* fix(langfuse): address PR #800 review feedback
P1 — Move trace forwarding off the daemon-internal run-close hook and
onto the message-persistence path. The original onTerminate hook ran
inside finish() the moment the SSE 'end' event was emitted, which is
*before* the web client's onDone handler refreshes project files and
PUTs producedFiles + final assistant content back to SQLite. Reading
SQLite at that moment routinely missed both. The fix: drop the runs.ts
hook entirely and trigger from PUT /api/projects/:id/conversations/:cid/
messages/:mid when the saved row carries a terminal runStatus. A
reportedRuns Set guards against the multiple PUT calls web makes per
turn (each retry / state update). Set entries auto-evict after the same
30 min TTL the runs map uses. Web persists a terminal-status message in
all three completion paths — onDone (succeeded), onError (failed), and
cancel (canceled) — so this catches every run shape.
P2 — postLangfuseBatch now parses the 207 Multi-Status response body.
Langfuse legacy ingestion always returns 207, and response.ok is true
for 207, so per-event validation errors used to slip through silently.
We now warn when body.errors is non-empty. Two new unit tests.
P2 — truncate() and the HARD_BATCH cap now compare UTF-8 byte length,
not String.length (which counts UTF-16 code units). A 4096-character
CJK prompt occupies 12 KB, well over the 8 KB input cap. truncate also
walks backwards to a UTF-8 leading byte so the cut never lands inside a
multi-byte codepoint. New unit test covers '设'.repeat(4096).
P2 — Spec R7 now lists the actual Langfuse trace deletion endpoint
(DELETE /api/public/traces/{traceId} for single, DELETE /api/public/traces
with body for batch). Verified by curl on us.cloud.langfuse.com:
DELETE /api/public/traces/X → 200; the path the original spec named
(POST /api/public/trace/X) returns 404. Reference link points at
langfuse.com/docs/administration/data-deletion.
P3 — Q4 (legacy ingestion vs OTel) moved from Open Questions to
Resolved Decisions. The implementation already commits to legacy and
the trade-off was discussed during design; the open-question status was
stale.
* feat(web): privacy consent surface + Settings → Privacy tab
Adds the user-facing half of the telemetry feature so the daemon-side
hook from PR #800 has something to talk to.
- AppConfig gains optional `installationId` (anonymous v4 uuid generated
on first opt-in; null after explicit decline; undefined when the user
has never seen the consent surface) and `telemetry: TelemetryConfig`
({metrics, content, artifactManifest}). syncConfigToDaemon round-trips
both fields so the bridge module sees the same prefs.
- SettingsDialog grows a Privacy section with two states. When the user
has never made a consent decision (typical first-run path), the
section renders the GDPR-aligned consent card: a kicker, the disclosure
body listing both metrics and conversation content as separate bullets,
and two equally-prominent buttons ("Share usage data" / "Don't share").
The Don't-share path keeps the app fully usable (core app must work
with all tracking declined). After a decision the same panel switches
to three independent toggles + the anonymous ID + a "Delete my data"
button that rotates the ID and turns everything off.
- App.tsx points the welcome modal at the new Privacy section so the
consent decision is the first thing a fresh installation sees.
- 17 i18n keys land in en + zh-CN + zh-TW with hand-translated copy,
and as English placeholders in the remaining 14 locales — enough for
the parity check to pass while leaving room for proper localisation
in a follow-up. Dict type updated.
- Minimal index.css for the consent card + toggle rows so the panel is
legible without depending on follow-up design polish.
Telemetry remains a no-op end-to-end until the user clicks Share usage
data: the daemon gate (prefs.metrics === true) keeps every code path
short-circuited otherwise.
* refactor(web): rebuild Privacy panel using project-native settings primitives
The first cut used custom .settings-privacy-* classes + raw HTML
checkboxes that didn't match any other Settings tab. Replace with the
shell other sections already use:
- settings-subsection containers with section-head + h4 + .hint
- seg-control / seg-btn pill toggles ("active" / "offline") for each of
the three telemetry preferences, mirroring NotificationsSection
- a 2-cell seg-control for the consent card so Share usage data and
Don't share carry identical visual weight (the GDPR equal-prominence
requirement that the previous accent / outline split missed)
- ghost button + readonly text input for the installation id row,
mirroring the API-key field pattern elsewhere
Drop the bespoke CSS block in favor of inheriting the existing
settings-section / seg-control / ghost styling. The only privacy-
specific style left is a tight definition list inside the consent card
for the metrics + content disclosure rows.
* refactor(web): use .toggle-row iOS switch for Privacy preferences
Active/offline pills (the seg-control single-cell pattern that
NotificationsSection uses) read awkwardly for a flat preference list.
Switch the three telemetry toggles to .toggle-row — the same control
NewProjectPanel uses for "speaker notes" / "animations": label + hint
on the left, iOS-style sliding switch on the right, full-row click
target. The consent card's two-button seg-control stays as-is — there
the equal-weight pill pair is exactly what GDPR equal-prominence wants.
* feat(web): standalone first-run privacy consent banner
Replaces the Settings-dialog-as-onboarding hack with a dedicated
bottom-right banner card that mounts whenever the user has never made
a privacy decision (cfg.installationId === undefined). The banner is
prominent (anchored to the corner with a soft shadow) but
non-blocking, mirrors cookie-consent UX, and shares the project's
panel styling — same .modal-elevated background, --radius-lg corners,
--shadow-lg lift.
Wiring:
- App.tsx imports PrivacyConsentModal and renders it at the root,
gated on installationId === undefined && !settingsOpen so it doesn't
double up with the Privacy tab's own consent card when Settings is
already showing.
- Share / Don't share both go through handleConfigPersist, so the
resulting installationId + telemetry prefs land in localStorage and
the daemon at the same time, reusing the existing autosave plumbing.
- The previous attempt that pinned the welcome SettingsDialog to the
Privacy section is reverted; onboarding now stays focused on agent
configuration, and the consent decision lives in its own surface.
* fix(web): keep privacy banner visible while Settings welcome modal is open
The banner gated itself on `!settingsOpen` to avoid double-rendering
with the Privacy tab's consent card. But the first-run path opens the
Settings welcome modal automatically when `onboardingCompleted=false`,
which fired immediately after bootstrap — so the banner flashed for a
moment and then vanished behind the modal backdrop.
Drop the `!settingsOpen` clause so the banner stays mounted whenever
the user has not yet made a privacy decision, and bump its z-index
above the modal backdrop (200 vs 100) so first-run users can actually
reach the consent buttons. The minor visual overlap with the Privacy
tab's own card is fine: clicking either copy resolves both surfaces.
* copy(privacy): soften consent button labels
Banner action buttons now read "Help improve Open Design" / "Not now"
(en, with hand translations in zh-CN / zh-TW and English placeholders
in the other 13 locales) instead of "Share usage data" / "Don't share".
The new wording aligns the affirmative action with the kicker copy
("Help us improve Open Design") and reads less alarming, while the
disclosure list above still names both data categories explicitly so
the consent stays informed under GDPR. The decline button stays as a
soft "Not now" rather than an aggressive "Don't share" so the reject
path doesn't read as hostile to the user.
No structural change — the two-cell seg-control still gives the buttons
identical visual weight, and the underlying side-effects are unchanged
(installationId is generated on Help / nulled on Not now, and the
telemetry prefs flip the same way).
* feat(telemetry): expand trace fields for evals & dataset construction
Each Langfuse trace now ships the full per-turn + per-install fact
sheet that the eval/dataset workflow needs, instead of only the bare
turn id + token count from before. Everything below is gated by
`prefs.metrics === true`; nothing here is content (those gates remain
separate).
Per-turn:
- model — first-class generation.model field, drives Langfuse cost
lookup and model-grouping in the UI; also mirrored in trace.metadata
and trace.tags so list-view filters work.
- reasoning — generation.modelParameters.{ reasoning } so the Model
Parameters card lights up; mirrored in metadata.
- skillId / designSystemId — metadata + tags, so dataset slices can
group by which skill/DS produced which output.
Per-process / build (constant within one daemon run, cached at start):
- appVersion / appChannel / packaged from app-version.ts
- nodeVersion (process.version), os (platform()), osRelease,
arch (os.arch())
- clientType — desktop vs web, derived from a new X-OD-Client header
the web layer sets in providers/daemon.ts (with a User-Agent sniff
fallback for third-party callers).
Plumbing:
- startChatRun stashes model / reasoning / skillId / designSystemId
on the run object alongside the existing userPrompt stash.
- POST /api/runs reads X-OD-Client and stores run.clientType.
- langfuse-bridge collects RuntimeInfo once per process and merges
per-run client carrier; ReportContext gains optional `turn` +
`runtime` blocks; existing fields stay backward compatible.
Spec gains a "Telemetry Fields Catalog" section enumerating every
field, its source, and the gate it lives under, so the eval team has a
single place to look up what's available without reading the trace
schema by example.
Tests:
- new langfuse-trace tests cover turn tags, runtime tags, generation
model/modelParameters promotion, modelParameters omission when
reasoning is unset, and metadata mirroring.
- langfuse-bridge gains an end-to-end "turn-level config" test that
threads model/reasoning/skill/DS/clientType + appVersion through
the bridge and asserts the Langfuse payload shape.
- existing tests adjusted to tolerate host-dependent os tag.
* copy(privacy): trim Share button to verb phrase only
"Help improve Open Design" overflowed the equal-width 2-cell
seg-control on the consent banner — the product name is already in
the kicker + headline above the buttons, so the button itself only
needs the verb phrase. Drop the product name from all locales:
- en: Help improve Open Design → Help improve
- zh-CN: 帮助改进 Open Design → 帮助改进
- zh-TW: 協助改進 Open Design → 協助改進
The decline button ("Not now" / "暂不" / "暫不") was already short, so
the two buttons now have comparable length and the equal-prominence
seg-control fits cleanly. Standalone Settings → Privacy panel uses
the same labels for consistency.
* fix(web): defer Settings welcome modal until privacy decision is made
Previously bootstrap raced two surfaces against each other on first
launch: the privacy consent banner (gated on installationId ===
undefined) and the Settings welcome modal (gated on
onboardingCompleted === false). The banner's higher z-index kept it
above the backdrop visually, but having two foreground surfaces at
once is still confusing UX.
Sequence them instead: bootstrap only opens the welcome modal when
the user has *already* resolved consent (installationId !== undefined).
Until then the banner owns the foreground alone. Once the user clicks
Help improve / Not now, the corresponding handler hands off to the
welcome modal if onboarding is still pending. End state matches what
it was before — just without the simultaneous-render flash.
* debug(privacy): log banner gate state to track sudden disappearance
Two console.log points to find which setCfg call (or stale bundle) is
flipping cfg.installationId from undefined to a value while the banner
is visible. To remove once the regression is reproduced.
* fix(privacy): keep installationId + telemetry out of localStorage
Daemon is now the single source of truth for the privacy decision.
Why this matters: the consent banner gates on
\`config.installationId === undefined\`, but loadConfig() merges
localStorage on top of the daemon's reply, so a stale uuid in
\`open-design:config\` (left over from a previous opt-in) was
re-hydrating the React state and immediately syncing back to the
daemon — defeating "Delete my data" and re-suppressing the banner
within milliseconds of every page load.
The deeper reason to fix it here, not just patch the gate: a privacy
identifier persisted in browser storage that the user can't see or
clear without DevTools is a compliance liability. Anything users can
revoke needs one canonical place to store it. Daemon \`app-config.json\`
already serves that role for everything else gated through
syncConfigToDaemon, so installationId + telemetry now ride that path
exclusively:
- saveConfig() strips both keys before writing localStorage.
- loadConfig() strips both keys when reading older stale payloads,
so existing installs migrate transparently on next launch.
- syncConfigToDaemon() / mergeDaemonConfig still round-trip them, so
the React state stays in sync with the daemon as before.
Net effect: clearing app-config.json (or hitting "Delete my data") now
fully resets the install identity, with no residual cohort key in
browser storage.
* feat(privacy): scrub secrets + PII from prompt/output before send
When prefs.content is on, daemon now runs the prompt and assistant
text through a regex scrubber (apps/daemon/src/redact.ts) before
posting to Langfuse. The scrubber is the simplest thing that gives
the user-facing copy a truthful claim — pure regex, zero new
dependencies, fully auditable in this Apache-2.0 repo (vs. pulling a
single-maintainer 5-month-old npm package into a core process).
Categories covered (each replaced with [REDACTED:<kind>]):
- Anthropic / OpenAI sk- keys (incl. proj/live/test/ant variants)
- Langfuse pk-lf- / sk-lf- (specific rule wins over generic sk-)
- GitHub gh[opsur]_ tokens
- AWS access key ids (AKIA + 16 uppercase)
- Google API keys (AIza + 35)
- Slack xox[abprs]- tokens
- Stripe live/test keys
- JWT header.payload.signature triples
- Bearer-header values (scheme word stays readable)
- Emails, IPv4, US-style phone numbers
- Credit cards — 13–19 digit runs that pass a Luhn check, so order ids
and unix-nanos timestamps that fail Luhn pass through unchanged
Not covered, stated openly in spec + i18n: names, postal addresses,
business-secret semantics, raw 40-hex tokens (too high a false-positive
cost for artifact slugs). Those would require an ML layer.
Wired in:
- apps/daemon/src/redact.ts — exports redactSecrets() +
redactSecretsWithCounts() helper for future audit-summary metadata.
- apps/daemon/src/langfuse-bridge.ts — runs both prompt and output
through redactSecrets() before they reach the trace builder.
- 18 unit tests cover every pattern plus negative cases (Luhn-failing
digit runs, out-of-range IPv4 octets, idempotence on re-redacted
text, ordinary prose passthrough).
- i18n privacyContentHint on en + zh-CN + zh-TW (plus 14 locale
placeholders) enumerates the categories so the consent disclosure
matches the implementation — the GDPR informed-consent requirement.
- spec gains a Pre-send Redaction subsection with the regex shape
table + intentional non-coverage list.
Drive-by: dropped the [privacy] debug logs that traced the now-fixed
bootstrap regression.
* fix(telemetry): make Langfuse reporting resilient
* feat(telemetry): nest Langfuse turn observations
* feat(telemetry): emit Langfuse tool spans
* fix(telemetry): report after finalized message writes
* fix(telemetry): honor persisted terminal status
* fix(web): let consent banner yield page clicks
* fix(telemetry): report current turn prompt only
|