Commit graph

4 commits

Author SHA1 Message Date
Patrick A
32fd5286b5
chore(e2e): improve test framework quality (#2305)
* chore(e2e): improve test framework quality

- Add lib/timeouts.ts with CI-scaled short/medium/long/xlong constants
- Add lib/playwright/mock-factory.ts to centralise standard localStorage,
  /api/agents, and /api/app-config mock setup; migrate critical-smoke and
  workspace-keyboard-flows to use applyStandardMocks()
- Delete empty lib/shared.ts placeholder
- Replace waitFor({ state: 'detached' }).catch(() => {}) with
  waitFor({ state: 'hidden' }) in all UI tests; 'hidden' resolves
  immediately when the element was never in the DOM, eliminating the
  silent error-swallowing catch
- Remove redundant .catch(() => false) from all isVisible() call sites
  since isVisible() never throws in Playwright
- Convert .waitFor().then(() => true).catch(() => false) guards in
  openDesignFile() to explicit try/catch blocks for clarity
- Simplify sendPrompt() in app.test.ts: replace the 3-attempt manual
  retry loop with a single fill + pressSequentially fallback; the core
  workaround for contenteditable unreliability is preserved but the
  loop structure is gone

* fix(e2e): guard routeMockAgents to GET only

routeMockAgents was intercepting all HTTP methods and returning the mock
fixture, silently swallowing any agent mutation requests. Mirror the
GET-only guard from routeAppConfig so writes fall through to the daemon.

* fix(e2e): address code review findings

- sendPrompt() in app.test.ts, workspace-keyboard-flows.test.ts,
  app-restoration.test.ts: drop fill() (unreliable on contenteditable,
  inputValue() always returns '' for them) and go straight to
  pressSequentially(), which types key-by-key and is authoritative
- Import T from timeouts.ts in app.test.ts and use T.short for the
  input/button waits, making the timeouts module non-dead

* fix(e2e): resolve adversarial review findings

- Revert sendPrompt to fill(): chat-composer-input is a textarea, not
  contenteditable; fill() is atomic and ~60x faster than pressSequentially
- Use T.medium in all waitForLoadingToClear calls: CI workers scale this
  to 20s automatically via the CI env var, eliminating cold-runner flakes
- Add T import to 6 files that needed it for T.medium
- Fix openDesignFile try/catch scope in app-manual-edit: previously the
  catch block only caught waitFor but click/expect errors were also swallowed;
  now only waitFor is inside try, real interaction failures propagate
- Fix regex escaping: .replace('.', '\\.') -> .replace(/\./g, '\\.') in
  app-manual-edit and app-design-files to handle multi-dot filenames
- Migrate entry-chrome-flows.test.ts to applyStandardMocks: it had the
  identical 3-call setup pattern as the factory but was not migrated
- Add GET method guard to project-management-flows app-config route handler,
  matching the pattern used by every other route handler in the suite
- Remove no-op 'as const' from timeouts.ts: Math.ceil returns number,
  not a literal, so the assertion had no effect
- Update e2e/AGENTS.md: remove deleted lib/shared.ts entry, document
  lib/timeouts.ts and lib/playwright/mock-factory.ts

* fix(e2e): scope openDesignFile try/catch to waitFor only

Move click and expect(preview).toBeVisible() outside the catch block so
that a regression in either open path (tab-click or file-list fallback)
fails loudly instead of being silently absorbed. The try now wraps only
the fileTabButton.waitFor existence probe; the subsequent click and final
assertion are unconditional.

---------

Co-authored-by: Patrick A <186436799+eefynet@users.noreply.github.com>
Co-authored-by: Patrick A <259201958+eefynet@users.noreply.github.com>
2026-05-23 00:24:32 +08:00
shangxinyu1
71044bd3d6
test(e2e): harden extended coverage state assertions (#2245)
* test(e2e): harden extended coverage contracts

* docs(testing): add e2e hardening status

* fix(web): persist artifact chips after daemon runs

* ci: install playwright browsers for e2e vitest

* Fix daemon run recovery across reloads

Pin daemon-created runs to assistant messages immediately so hard reloads before the create response can reattach.

Replay terminal and active run events from the beginning on reload so restored turns keep assistant text, thinking events, produced files, and artifacts.

Fixes #2366

Fixes #2368

Fixes #2371

* test(e2e): preserve fake runtime selection across reload

* fix(web): scope daemon run recovery to daemon mode

* fix(e2e): remove duplicate delayed smoke flag

* fix(web): scope replay artifact recovery to current run

* fix(daemon): remove duplicate run-create pin
2026-05-20 16:21:01 +08:00
PerishFire
976edaf38e
test: harden e2e smoke and release reports (#1140)
* test: harden e2e inspect specs

* test: wire e2e release reports

* chore: bump packaged beta base to 0.6.1

* test: run release smoke vitest directly

* test: add suite-owned tools-dev lifecycle

* ci: harden stable release packaging

* fix(release,e2e): gate stable signing on verify and harden suite cleanup

- restore `needs: [metadata, verify]` on the stable release `build_mac`,
  `build_mac_intel`, `build_win`, and `build_linux` jobs so Apple
  signing/notarization and Windows release builds cannot run before
  pnpm guard, typecheck, and layout checks complete on the metadata commit.
- in `runToolsDevSuite`, drop the `started` flag and always attempt
  `stopToolsDevWeb` in `finally`; record stop errors in diagnostics, and
  when the test body succeeded, escalate the stop failure to the suite
  result and rethrow — so orphan daemon/web processes from an interrupted
  `startToolsDevWeb` or a broken shutdown can no longer pass silently.

Addresses PR #1140 review feedback from lefarcen and mrcfps.
2026-05-11 13:11:16 +08:00
PerishFire
f1cdb2844a
test(e2e): gate beta packaged runtime (#637)
* test(e2e): gate beta mac packaged runtime

* test(e2e): separate ui automation layout

* test(e2e): move localized content coverage

* chore(release): prepare packaged 0.4.1 beta validation

* test(e2e): keep ui lane playwright-only

* fix(web): keep chat recoverable after conversation load failure

* fix(desktop): honor native mac quit
2026-05-06 17:44:29 +08:00