Product analytics and activation

Enable the V1 features

These three switches are independent. AI usage controls default to enabled; product analytics and activation default to disabled:

PRODUCT_ANALYTICS_ENABLED=false
ACTIVATION_ENABLED=false
AI_USAGE_CONTROLS_ENABLED=true

pnpm run init offers each switch in Advanced → Features and integrations and writes all three to .env.local. Fresh Simple setup enables AI usage controls and leaves analytics and activation disabled. Reconfiguration loads and preserves existing true or false selections in either mode; Advanced mode lets you change them. When omitted, AI_USAGE_CONTROLS_ENABLED also defaults to enabled at runtime; explicitly set it to false to opt out. The summary shows each selection. Docker Compose forwards the same values to the app and both workers.

Use the current fresh-install SQL manifest. This feature does not provide an in-place upgrade script. pnpm run init seeds the maintain-product-insights internal job to run every minute. After changing the flags, restart the app and worker, then run that job once from the admin job interface. It applies the analytics/activation switches and current privacy-policy version to the private database settings. Collection starts after this succeeds. Keep the critical worker lane running and enable job failure notifications.

The dashboard links to AI usage & controls; the organization overview links to the workspace version. The platform overview links to Product analytics. Pages return 404 when their feature is disabled. The checklist appears at the end of the private and organization overview, with a per-member hide/show preference. RAG steps require an available embedding model, and the teammate step requires B2B mode and invitation permission.

Measurement definitions

config/product-analytics.ts owns the activation event, ordered funnel, windows, retention and checklist definitions. The default activation milestone is a successfully completed AI answer with provider-reported usage.

Events come from committed account membership, onboarding, AI request, document readiness and normalized payment writes. No browser API accepts arbitrary event names, properties or completion flags. Duplicate events compact into one Account/actor/event/UTC-day row with first and latest timestamps, preserving return-use boundaries within a day. Rolled-back business outcomes never appear. Reports return aggregate counts, not member activity feeds. Zero denominators display “Not enough data.”

Observation starts when consent is recorded for an existing membership, a consenting actor joins an Account, or the actor loads an overview. Earlier product activity is not backfilled. This means the metrics describe the observed, consenting population, not all historical signups. A first-milestone funnel deliberately excludes accounts whose first milestones occurred in a different order; use adoption alongside it.

Consent, retention and erasure

Measurement requires user_consents.analytics to be accepted against complianceConfig.privacyPolicyVersion. Consent withdrawal removes that actor's analytics events and enrollment markers transactionally. Reports recheck current consent and policy version. The operational checklist and AI accounting remain available independently of analytics consent.

Analytics stores Account ID, actor ID, milestone and timestamps. It never copies prompts, answers, document content, IP addresses or arbitrary user properties. Events expire after the configured 90 days in bounded worker batches. Minimal first-observation markers remain until consent withdrawal, subject erasure or Account deletion to prevent old accounts from reappearing as new cohorts. Checklist milestones remain for the life of the Account.

GDPR exports include analytics events/enrollment, checklist preferences and personal-Account milestones, plus actor-linked AI policies, counters, operations, alerts and changes. Exports fail explicitly for operator review above the configured row/byte bounds. Subject foreign keys erase or remove attribution; Account deletion cascades through the private tables. Policy audit and monthly summaries expire after 365 days by default. Minimal operation keys and unresolved usage receipts survive ordinary retention because deleting them could permit a second debit; Account deletion removes them.

AI allowances and policies

config/ai-usage-controls.ts owns bounds, alert thresholds, retry policy and retention. Owners and administrators receive ai:manage; custom roles can be granted it through the existing role editor. ai:use remains the permission to run AI. Policy changes require the normal authenticated CSRF boundary and adminEscalation step-up, with an atomic audit row and optimistic version check. Members see their own member allowance; managers can edit bounded pages of members.

• Empty monthly allowance means no allowance limit. Zero blocks new work. A member's effective allowance is constrained by both the member and Account limits.
• Months start at midnight UTC. Usage is assigned to the month when the operation was admitted, even if completion arrives after the boundary.
• These are stop-new-work thresholds. An admitted request may cross a threshold; this is not a reserved-token or guaranteed never-exceed budget.
• Credits remain separate. Every provider-reported input/output or embedding token costs one credit, using the existing ledger and semantic idempotency keys. No reserve/refund flow is introduced.
• Allowed agents and models are enforced on the server. Model fallback is restricted to the intersection of the Account policy and the agent's supported models. Output limits can only reduce the global maximum.
• Chat, query embeddings and document embeddings share Account admission coordination. Document workers retry temporary contention. Provider-confirmed usage remains accounted for after cancellation or error.
• Notifications are deduplicated at 80% and 100% per subject/month. Account alerts go to current ai:manage members; member alerts go to the affected member. Background notifications use the configured default locale because the profile has no persisted locale preference.

Disabling the optional controls stops new policy enforcement and hides their UI. The worker continues draining and reconciling already-recorded debts; disabling a flag must never forgive provider usage.

Recover uncertain embedding usage

The normal maintain-product-insights job settles completed embedding receipts with bounded retries. It uses the same rag-search/rag-ingestion ledger keys as the request path, so a process crash or duplicate worker cannot charge twice. It alerts through a failed job if usage is unresolved or settlement retries are exhausted.

If some batches succeed and another fails without authoritative usage, the known tokens count toward allowances but the operation stays unresolved. It blocks new work until the missing usage is confirmed. Do not expire the fence, guess tokens, or release it solely because a timeout elapsed.

After verifying the total with the provider, run the internal reconcile-ai-embedding handler from the protected admin job interface with:

{
  "operation_key": "search:00000000-0000-4000-8000-000000000001",
  "tokens": 120,
  "provider_reference_digest": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
  "provider_usage_confirmed": true
}

Use the actual existing operation key (search:<operation UUID> or embedding:<document UUID>:<revision>), confirmed total and SHA-256 digest of the provider evidence reference. The example values are placeholders. Raw provider responses and credentials must not be stored in the job. A confirmed zero is valid only with evidence that no tokens were consumed. Reconciliation never calls the provider again, and an already completed total cannot be changed. Chat recovery continues through settle-ai-usage and its existing confirmed-admission recovery input.

Verification

node scripts/qa/prove-product-insights.mjs installs and checks all four disposable Database/Auth compositions. It tests private-table denial, foreign Accounts, active identity, manager/member permissions, version conflicts, concurrent admission, exact replay-safe accounting, partial recovery, UTC period attribution, current consent, cohort definitions, exports and erasure. This is local-only evidence; it does not certify hosted infrastructure performance.

The proof also times a report over 2,000 synthetic Accounts and 28,000 daily events. Set QA_INSIGHTS_QUERY_PLAN=true to include a sanitized execution-plan summary. A single composition can be selected, for example node scripts/qa/prove-product-insights.mjs neon:better_auth.

Run pnpm run qa:local:insights -- --composition=neon:better_auth for the enabled-feature browser profile. It exercises member/manager authorization, policy changes, admission limits, consent, activation preferences and the platform report. Accessibility scans cover every configured locale and both themes at 320 and 1440 pixels; responsive checks also cover 375, 768 and 1280 pixels. Run database and browser profiles sequentially because they own disposable infrastructure. These checks do not replace hosted certification or production load testing.