PrismBot RAG Implementation

Raw documents are noisy. PDFs carry headers, footers, page numbers, and navigation furniture that mean nothing to an embedding model but dilute every vector they land in. We run an LLM preprocessing pass over each document first, stripping that noise and normalizing content into a consistent structure.

Chunking then happens against the cleaned structure rather than a fixed character count. Each chunk carries its section and subsection headers inline, so a passage about renewal terms arrives already labeled with the product and policy section it belongs to. The chunk is self-describing even when it's retrieved alone, with none of its neighbors for company.

This is the least glamorous stage in the pipeline and the one with the largest effect on output quality. Every improvement further down compounds on top of it.

A user asking "how long do I have to send something back" and a document saying "returns must be initiated within 30 days of delivery" share almost no vocabulary. Single-query similarity search misses that match often enough to make an assistant feel unreliable, and the failure is invisible, because the model answers confidently from whatever it did retrieve.

We expand each incoming query into multiple rewritten variants, run those searches in parallel against the vector store, and merge the results. Reranking with MMR then drops near-duplicate passages that would otherwise burn context window without adding information, and promotes passages covering different facets of the question.

The combination produced roughly four times the answer relevance of single-query top-K retrieval.

Vectors live in PostgreSQL with the pgvector extension rather than in a dedicated vector database. Keeping embeddings in the same database as tenant records, documents, and permissions means tenant isolation is enforced by the same constraints protecting everything else, instead of by a second system that has to be kept in sync with the first.

An HNSW index handles approximate nearest neighbor search, and every query filters by tenant namespace before the similarity search runs. Retrieval stays under 300ms as the corpus grows.

Choosing Postgres over a specialized store costs some raw throughput at the top end. It buys operational simplicity: one database to back up, one place where access rules live, and no window in which a tenant's vectors exist inside a system that doesn't know the tenant boundary.

One prompt doing retrieval, context assembly, and answer generation is quick to write and hard to fix. When answers degrade there's no way to tell which part of the chain caused it, so debugging turns into prompt roulette.

We split the work with LangGraph into agents with distinct responsibilities. One owns retrieval and query expansion. One enriches retrieved context and resolves references against conversation history, so "what about the enterprise plan" still works three turns in. One generates the response. One evaluates whether the answer is grounded in the retrieved passages. Each agent is independently testable and independently replaceable.

That structure is also what makes escalation possible at all. The grounding check needs somewhere to hand off to when it decides the retrieved context doesn't support an answer, and a monolithic chain has no such seam.

Chat runs over WebSockets with Socket.IO, so web and WhatsApp conversations share one server-side state model. A conversation that starts in the website widget doesn't lose its history because the transport changed underneath it.

Escalation is a designed path, not an error handler. When the grounding check fails or someone asks for a person, the conversation moves into a human agent queue, and the agent picks it up with the full transcript and the passages the assistant retrieved. They start with context instead of asking the user to repeat themselves. Push notifications and email alerts fire on handoff, so nobody sits in a queue that nobody is watching.

Every escalation is also a signal about the corpus. A cluster of handoffs on one topic usually means a document is missing, not that the model is underperforming.