Testing

The boilerplate includes a comprehensive test suite with three layers: Vitest (unit tests), Playwright (E2E, accessibility, responsive), and Lighthouse CI (performance auditing).

Test Commands

pnpm run test              # Vitest unit tests
pnpm run test:watch        # Vitest watch mode
pnpm run test:e2e:auth     # Anonymous auth and protected-route smoke tests
pnpm run test:e2e:oauth    # Deterministic provider/locale/PKCE/consent contracts
pnpm run test:e2e:api      # Protected-API anonymous authorization contracts
pnpm run test:e2e:personas # Authenticated role, tenant-isolation, and dashboard axe smoke
pnpm run test:e2e:private  # Provider-free private dashboard and GDPR export contracts
pnpm run test:e2e:org      # Organization dashboard owner/admin/member contracts
pnpm run test:e2e:admin    # Platform-admin routes, CRUD/actions, validation, audit, and denials
pnpm run test:e2e:billing  # Billing scope, checkout, portal, webhook, refund, dispute, and trial contracts
pnpm run qa:local:gate     # Disposable Supabase + pgTAP + four personas + Playwright
pnpm run qa:local:private  # Focused disposable chat/SSE + documents/RAG gate
pnpm run qa:local:org      # Focused disposable organization-dashboard gate
pnpm run qa:local:admin    # Focused disposable platform-admin gate
pnpm run qa:local:billing  # Focused disposable billing + deterministic Stripe gate
pnpm run qa:local:phase6   # Production non-functional release gate
pnpm run qa:local:phase6:update-visuals # Explicit visual-baseline refresh
pnpm run qa:local:phase7   # Focused browser-driven admin CRUD journeys
pnpm run test:cross-browser # Chromium, Firefox, and WebKit matrix
pnpm run test:visual       # Compare committed visual baselines
pnpm run test:a11y         # WCAG 2.2 AA accessibility tests (Playwright + axe-core)
pnpm run test:responsive   # Responsive design tests (320/375/768/1280/1440px)
pnpm run lighthouse:mobile # Lighthouse CI audit
pnpm run lighthouse:phase6 # Hard desktop Lighthouse release gate
pnpm run lint:ci           # ESLint errors only (current CI gate)
pnpm run lint:strict       # ESLint with zero warnings (target gate)
pnpm run lint:debt         # Historical warning debt report
pnpm run audit:guardrails  # Project-specific anti-regression checks
pnpm run audit:api-patterns # API security wrapper coverage
pnpm run audit:rls         # Static RLS coverage
pnpm run audit:load        # Configurable API load smoke test
pnpm run test:staging      # Real staging readiness probes
pnpm run db:types:check    # Supabase schema/type coverage

Unit Tests (Vitest)

Tests in __tests__/ configured with vitest.config.ts (node environment).

E2E Tests (Playwright)

Tests in tests/ configured with playwright.config.ts (Chromium, 30s timeout, parallel on CI).

Playwright validates QA_TARGET, QA_BASE_URL, QA_AUTHENTICATED_SETUP, and QA_PERSONA_SETUP before registering authenticated projects. Anonymous tests default to 127.0.0.1. Authenticated runs require an exact pre-provisioned E2E_TEST_USER_ID/E2E_TEST_EMAIL match and isolated Supabase QA credentials; hosted staging also requires a separately repeated QA_APPROVED_SUPABASE_ORIGIN. The four-persona suite is local-only because it provisions and revokes Auth state. Production/indexable targets, hosted persona targets, mismatched origins, privileged publishable keys, and live Stripe keys fail closed before an admin client is created.

pnpm run qa:local:gate is the disposable authenticated gate. With Docker running, it starts the pinned local Supabase stack, performs a guarded reset/seed, runs pgTAP and database lint, provisions four account-centric actors through the Admin Auth API, and records the application-rendered magic-link message through a run-scoped local noop adapter. Playwright proves both delivered sign-in paths, that ordinary local logout preserves a second session, and that two-session scope: 'others' revocation is immediately enforced by application requests revalidated through getUser(), then runs role/tenant/dashboard accessibility checks. Already-issued Supabase access JWTs used directly against the Data API may remain valid until expiry; the gate does not claim universal immediate JWT invalidation. Mutating Auth journeys use dedicated states. The app binds to 127.0.0.1, the Supabase origin must be loopback, and the run must be non-indexable development. Privileged tests use a non-persistent reporter; token-bearing artifacts stay disabled, and generated browser state plus temporary email/output directories are removed on normal completion or handled interruption before the post-gate secret scan. The exact local volume is then torn down. The gate never operates on a linked or hosted project.

The gate starts a secret-free OpenRouter-compatible fixture on 127.0.0.1. The application still uses its real LlmClient, chat, document, Storage, and Supabase paths. The fixture rejects chat or embedding requests that omit ZDR/data-collection-deny routing, strict parameter support, attribution headers, the safe key, or canonical gateway model IDs. Playwright proves those contracts with chat accounting/resilience, transport persistence, deterministic embeddings, exact provider-token debit, tenant isolation, and document cleanup. Stalled work is bounded by aiConfig.streamTimeoutMs or aiConfig.embeddingRequestTimeoutMs without false success accounting. Hostile parent provider credentials and URLs are removed from the child environment, and generic projects exclude the provider-backed specs.

pnpm run qa:local:org runs the focused organization-dashboard Phase 3 slice with the same guarded disposable stack. It covers all nine route contracts across owner, admin, and member personas; invitation, role, member, API-key, settings, and deletion mutations; analytics and billing account isolation; owner-only billing/destructive controls; and WCAG 2.2 A/AA plus 320/768/1280px overflow checks on all eight rendered destinations. It then repeats the authenticated read, mutation, and dynamic-resource API contracts. Use pnpm run test:e2e:org only with an already provisioned, validated persona target.

pnpm run qa:local:admin runs the focused platform-admin Phase 4 slice. Its hermetic child enables logs, referrals, and affiliates, renders all 43 admin list/new/detail/edit pages with worker-owned IDs, covers positive CRUD/actions plus malformed and unknown-resource failures, and proves page/API denial for member, workspace-admin, and workspace-owner personas. The same profile asserts audit rows and the real error-log writer's secret redaction and salted IP hashing. Stripe subscription transitions remain provider-free: a deterministic domain contract proves provider call, local mirror, and audit alignment; test-mode Stripe journeys belong to the external-integration gate. Use pnpm run test:e2e:admin only with an already provisioned, validated persona target and matching feature flags.

pnpm run qa:local:billing runs the focused billing Phase 5 slice. It keeps the application routes, Stripe signature verification, Stripe SDK, Supabase RPCs, and ledger writes intact; only Stripe's remote HTTP boundary is replaced by a fixed, secret-free loopback provider that cannot inherit live credentials. The profile proves owner-only provider contact, exact price/product binding, positive credit-pack checkout, fail-closed license checkout when the configured model disables it, portal creation, safe provider errors/timeouts, webhook replay/concurrency/order, trial conversion, invoice suppression, partial/full refunds, lost-dispute reversal, and non-negative idempotent credit accounting. Use pnpm run test:e2e:billing only with an already provisioned, validated persona target and the exact deterministic-provider marker.

pnpm run qa:local:phase6 runs the non-functional release profile against a production Webpack build and the generated Serwist worker. It reuses the guarded disposable Supabase/persona lifecycle, then covers 14 stable public pages in both themes, docs/offline, all 58 owned private/organization/admin destinations at five viewports, eight reviewed visual baselines, Chromium/Firefox/WebKit, consent/GPC/PWA/offline/push behavior, API/document runtime-cache isolation, public and authenticated navigation/transfer/CLS budgets, persona-specific PostgreSQL query ceilings, 15 hard Lighthouse runs, and a rate-bounded health load smoke. The gate remains loopback-only and disables persistent privileged browser artifacts. Use pnpm run qa:local:phase6:update-visuals only when intentionally reviewing every changed baseline.

pnpm run qa:local:phase7 runs the focused browser-driven platform-admin CRUD profile. It currently exercises worker-owned jobs, job handlers, blog categories, and localized tags through the real forms, menus, confirmation dialogs, CSRF-protected APIs, and exact database cleanup checks. The taxonomy journeys also prove that in-use categories and tags cannot be deleted. Method + canonical-route rate-limit scopes prevent cross-domain bucket collisions while sensitive mutations retain the real 5/min/route tier. The same spec is included in the Phase 4 admin gate and the default disposable gate; CMS/blog page publishing, media/Storage, and cron UI journeys remain in progress.

Lighthouse CI

lighthouserc.js uses the desktop preset and audits /fr-FR, pricing, login, privacy, and /docs three times each. pnpm run lighthouse:phase6 enforces the thresholds below as hard errors against the median run; pnpm run lighthouse:mobile remains the separate mobile-emulation command. The localhost profile explicitly excludes only diagnostics that cannot be represented there (intentional noindex/HTTP and antivirus-injected resources); production indexability, HTTPS, and CSP stay covered by configuration/security contracts.

Quality Audit Commands

pnpm run audit:signatures  # registry signature and attestation verification
pnpm run audit:security    # pnpm vulnerability scan
pnpm run audit:secrets     # secretlint (secret detection in code)
pnpm run audit:guardrails  # project guardrails: direct credits writes, env leakage, schema mirrors
pnpm run audit:api-patterns # API security wrappers and route-pattern exceptions
pnpm run audit:rls         # full-schema RLS enabled/policy coverage
pnpm run audit:load        # HTTP load smoke test against a configured external target
pnpm run db:types:check    # generated Supabase types cover full schema tables/views
pnpm run audit:deps        # knip (unused dependencies/exports)
pnpm run audit:circular    # dependency-cruiser (circular imports)
pnpm run audit:bundle      # Bundle size analysis (webpack build; analyzer does not emit reports under Turbopack)

GitHub CI

.github/workflows/ci.yml runs on pull requests and pushes to main. It performs a frozen pnpm install, registry signature and vulnerability audits, then gates typecheck, lint:ci, lint:strict, project guardrails, API-pattern coverage, RLS coverage, Supabase type coverage, unit tests, production build, anonymous authentication/access smoke tests, accessibility, responsive checks, and a post-browser secret scan. A separate disposable-local job runs pgTAP plus the four authenticated persona/ACL/dashboard checks against a fresh loopback Supabase stack. Historical warning visibility lives in pnpm run lint:debt; that report is intentionally separate from the zero-warning gate.

Staging and External Checks

pnpm run test:staging targets a real deployed environment through STAGING_BASE_URL (or NEXT_PUBLIC_APP_URL) and real provider credentials. It probes /api/health, Stripe webhook endpoint and required events, Supabase Postgres schema, cron extensions, storage buckets, enabled jobs, GDPR deletion proof columns, Upstash Redis, and the configured Brevo/Mailjet account. Safe job execution is optional and requires STAGING_RUN_SAFE_JOB=true plus STAGING_SAFE_JOB_NAME.

pnpm run audit:load is a lightweight external API load smoke test. Configure LOAD_TEST_BASE_URL, LOAD_TEST_DURATION_SECONDS, LOAD_TEST_CONCURRENCY, optional LOAD_TEST_REQUESTS_PER_SECOND, LOAD_TEST_P95_MS, and optionally LOAD_TEST_ENDPOINTS JSON to cover authenticated or critical API paths. A rate cap is recommended when the goal is latency rather than rate-limit saturation. It complements, but does not replace, a third-party security/performance review.