RAG & Documents

RAG Document Chat

Upload documents (TXT, MD, PDF) and chat with them using the Knowledge Base AI agent. Documents are chunked and embedded in bounded batches through the shared LlmClient using either direct OpenAI or OpenRouter, and vector similarity search runs inside Postgres via the membership-enforced match_document_chunks_text SECURITY DEFINER RPC over a pgvector HNSW index. The match cutoff is configurable via embeddingConfig.ragMatchThreshold (default 0.1 in config/ai.ts). Credits are deducted 1:1 with provider-reported embedding tokens. PDF text extraction uses unpdf.

RAG Agent Integration

The Knowledge Base agent (lib/ai/agents/rag/index.ts) is fully integrated into the AI chat system. It uses a pre-fetch pattern — the stream route calls prepareRAGContext() before streaming, which caches the document context. Then buildMessages() injects it into the user message.

1. Stream route calls agent.prepareRAGContext(accountId, message) before streaming
2. Fetches all chunks for the account and generates the query embedding through the selected direct/OpenRouter route
3. Runs vector similarity search via the match_document_chunks_text Postgres RPC (pgvector HNSW index, membership enforced inside the function — no raw connection pool)
4. Filters by threshold (configurable via embeddingConfig.ragMatchThreshold, default 0.1) and keeps top 5 matches
5. buildMessages() injects cached context into the user message (not system prompt) wrapped in XML tags (<document_chunk>) for prompt injection protection
6. The LLM responds using the document context, citing sources like [Document 1]

The agent is registered in lib/ai/agents/index.ts and appears in the chat agent selector dropdown with a rose/pink color scheme.

Configurable Embedding Models

The embedding model used for document processing is configurable in config/ai.ts via the embeddingConfig object:

To change the default model, update embeddingConfig.defaultModel in config/ai.ts. The model, model provider, transport, reported upstream provider, and exact token count are stored in document/credit metadata for traceability. Embedding calls use batches of at most 96 inputs, bounded concurrency, schema/dimension/index validation, and the selected AI_LLM_TRANSPORT. OpenRouter requests carry the same mandatory ZDR/data-collection-deny routing as chat.

Credit Deduction on Upload

1 credit = 1 LLM token. Credits are deducted from the account based on the actual usage.total_tokens reported by the selected direct/OpenRouter embeddings route across all chunks of the document, summed and decremented in a single decrement_credits RPC after the loop completes.

Total credits = sum of usage.total_tokens for each chunk embedding call

For example, a 10-page document producing 50 chunks of ~250 tokens each will cost roughly 12,500 credits, billed exactly to the token. A pre-flight check ensures the account has at least aiConfig.minCreditsRequired (default: 100) before calling the selected provider route; the post-loop decrement is best-effort — if a concurrent request races the balance below the required tokens, the failure is logged via logError but the document remains indexed.

Knowledge Base Page

The document manager is accessible at /private-dashboard/documents and appears in the sidebar as "Knowledge Base" (right below "AI Agents"). The page provides:

• Drag-and-drop file upload with progress tracking
• Real-time status polling (pending → processing → ready)
• Document list with status badges, chunk counts, file sizes, and exact provider-token balance refresh
• Delete with confirmation and full cleanup (chunks + storage)

API Routes

File Structure

lib/rag/
└── index.ts                 # chunkText, generateEmbedding, processDocument,
                             # searchDocuments (match_document_chunks_text RPC),
                             # buildRAGContext, parsePdf (unpdf)

lib/ai/agents/rag/
└── index.ts                 # RAGAgent class (extends BaseAgent)
                             # prepareRAGContext() + buildMessages() pattern

config/ai.ts
├── embeddingModels          # Available embedding models with dimensions + cost
└── embeddingConfig          # Default embedding model (credits are 1:1 with tokens, no flat rate)

config/ai-server.ts          # Validated server-only transport, credentials, and base URLs
lib/ai/llm-client.ts         # Sole chat/embedding provider boundary

app/api/documents/
├── route.ts                 # GET (list), POST (upload), DELETE (remove)
└── [id]/route.ts            # GET (details + chunk summaries)

app/[locale]/(private)/private-dashboard/documents/
└── page.tsx                 # Knowledge Base page (server component)

components/documents/
└── document-manager.tsx     # Upload UI with drag-and-drop + status polling