QA Automations

The QA system is a hermetic, account-aware release gate. It tests the real Next.js application against disposable local services, provisions explicit account and role fixtures, replaces only remote provider boundaries, and removes the generated state when the run ends.

Use this page when you need to run the gates, understand what they prove, or add coverage for a new route. For the lower-level Vitest, Playwright, Lighthouse, and audit catalog, see Testing. The complete implementation ledger and remaining work live in QATests.md at the repository root.

Start here

The repository requires its pinned Node 24.18.1 and pnpm 11.20.0 versions. Docker must be running because the main gate starts a local Supabase stack.

# Recommended comprehensive local gate
pnpm run qa:local:gate

# Inspect or stop the local QA stack when troubleshooting
pnpm run qa:local:status
pnpm run qa:local:stop

qa:local:gate is self-contained. Focused qa:local:* profiles use the same guarded lifecycle and are faster when you are working in one domain. Direct test:e2e:* commands generally expect an already running application and, for authenticated suites, pre-provisioned validated personas.

How the disposable gate works

1. Validate the target before creating privileged clients. Zod-validated QA configuration rejects production or indexable targets, hosted Supabase origins under local intent, mismatched approved origins, privileged publishable keys, live Stripe keys, and unsafe email/provider settings.

2. Start clean local infrastructure. The runner starts the pinned Supabase CLI stack on loopback, replays the schema and every migration, loads deterministic seed data, and starts the Next.js application on port 3777 with test-only credentials.

3. Prove the database contract first. pgTAP and database lint verify RLS, Data API grants, constraints, RPC permissions, account isolation, credit-ledger behavior, cron objects, and fresh-schema/migration consistency.

4. Provision account-centric personas. Setup creates explicit profiles, Accounts, Memberships, roles, access states, and isolated browser storage states. Mutable tests receive worker-owned users and Accounts so parallel projects cannot share state accidentally.

5. Run application contracts. Vitest checks domain behavior; Playwright drives API and browser journeys through real routes; axe-core, viewport, visual, cross-browser, PWA, performance, and reliability checks run in their assigned profiles.

6. Collect bounded diagnostics and clean exactly what the run owns. Failures include a reproduction command and secret-safe evidence. Privileged runs do not persist traces, screenshots, videos, raw URLs, headers, bodies, or credentials. Cleanup removes exact worker/run resources, browser state, temporary email/provider files, and the disposable local volume—even after handled failure or interruption.

Command map

Self-contained local profiles

Lower-level and already-provisioned commands

What the system proves

The automation is layered so a failure is caught as close as possible to its source:

Every configured locale receives navigation/render smoke coverage. Functional journeys use a deterministic default locale, with locale-sensitive assertions repeated across French and English regional variants.

Accounts, personas, and isolation

The fixtures follow the product architecture: domain data attaches to an Account, not directly to a User. A persona is valid only when its profile, Account, Membership, role, billing/access state, and active-account selection are explicit.

The core local matrix includes ordinary members, workspace admins, workspace owners, and a platform admin, plus anonymous and specialized states such as zero credits, disabled access, pending invitations, unrelated workspaces, active subscriptions, and licenses. Tests use run and worker identifiers for mutable records, seed dynamic resource IDs instead of hardcoding UUIDs, and assert both wrong-role denial and cross-account denial.

This is why the gate can run state-changing projects in parallel without weakening tenant-isolation assertions or allowing one test to depend on another test's mutation.

Route manifests prevent silent coverage drift

Two typed manifests classify every page and route handler:

• tests/manifests/pages.ts records the surface, persona, locale, feature gate, and expected page outcome.
• tests/manifests/api-routes.ts records exported/effective methods, domain, security wrapper, access class, mutation/CSRF behavior, account scope, validation, idempotency, side effects, and expected errors.

The route-drift guard scans app/**/page.tsx and app/**/route.ts. A new or changed route fails until its manifest entry says how it is tested, redirected, feature-gated, or intentionally excluded with a reason. This turns missing QA classification into a review-time failure instead of a future production surprise.

Third-party boundaries stay deterministic

Core browser journeys do not stub the application's own API. They keep the real route, domain logic, Supabase writes, Storage behavior, security middleware, and browser UI, then replace only the remote dependency:

Optional staging checks use dedicated test-mode projects and pre-provisioned identities. Production permits only read-only synthetic checks such as health, public availability, TLS, headers, and a non-mutating login render.

Reading failures and reports

QA output identifies the affected surface, route or domain, persona/profile, pass/fail/skip/flaky count, and—where relevant—accessibility impact, performance delta, or visual diff. Stable prefixes such as AUTH-*, ACL-*, ORG-*, ADM-*, API-*, DB-*, BILL-*, AI-*, A11Y-*, and PERF-* make failures searchable.

Retries collect evidence; they do not convert flaky behavior into a pass. Start with the focused reproduction command printed by the failing gate, then use pnpm run qa:local:status to confirm the local stack if the failure occurred during setup. Do not copy credentials, callback URLs, browser state, or provider payloads into an issue.

Adding QA for a change

1. Update the page or API manifest whenever a route is added, removed, or changes its effective methods or access behavior.
2. Put pure domain behavior in Vitest and database authorization/invariants in pgTAP.
3. Add API contracts for validation, auth, CSRF, role, account isolation, idempotency, and documented failures.
4. Add a browser journey for user-visible behavior; use accessible-name locators and assert persisted outcomes rather than implementation details.
5. Give state-changing tests worker-owned Accounts and resources, with exact fail-closed cleanup.
6. Run the smallest relevant focused profile while developing, then pnpm run qa:local:gate before merge.

No test should use fixed sleeps, depend on another test's mutation, auto-approve a snapshot, weaken a rate limit, or target production. Expected skips and allowlists need a reason, owner, and expiry instead of becoming permanent gaps.