03 · RAG / Knowledge Systems
Live in productionInsurance RAG Knowledge System
Retrieval assistants over insurance documents. I built the embedding pipelines, prompts, and tool schemas, with Pinecone and FAISS, and a multi-model chatbot on the eSanad quote flow.
- Role
- Built the pipelines and prompts
- Scope
- Ingestion, embeddings, retrieval, chat
- Stack
- Next.js, LangChain, LangGraph
Client details are confidential. No client data is shown.
Overview
These assistants answer from insurance documents. I wrote the embedding pipeline, the prompts, the system instructions, and the function-calling schemas. LangChain and LangGraph run the flow. Pinecone and FAISS hold the vectors. One path is a local document question-answering stack with FAISS and Ollama. Another is the eSanad chatbot, which sits on a quote flow that compares providers and can call more than one model.
The problem
The documents are the source of the answer. A chat box that does not retrieve them will sound confident and be wrong. Sending the whole file on every question also burns tokens. Department and product boundaries matter, because a claims passage is not a quote passage.
What I built
- The ingestion and embedding path, and the prompts that wrap a retrieved passage.
- Pinecone and FAISS indexes, used where each store fit the deployment.
- LangChain and LangGraph orchestration, including tool schemas I wrote.
- A local RAG path on FAISS and Ollama for document question answering.
- The eSanad chatbot, with multi-model support, next to the Node.js and MongoDB aggregator.
- Token-saving context: the model sees the retrieved slice, not the archive.
Architecture
Documents are parsed and embedded, then stored in Pinecone or FAISS. A question retrieves the nearest passages. The prompt is assembled from those passages and the system instructions. The model answers from that context. On eSanad, the same pattern sits beside quote comparison across providers.
Diagram
Key engineering decisions
Pinecone where the index is a hosted service, FAISS where the index can live next to the app, including the local Ollama path. The orchestrator does not care which store returned the passages.
I kept the prompt on the retrieved text. That is both the grounding and the cost control. Function-calling schemas are explicit, so a tool is a named operation and not a paragraph the model is supposed to infer.
Hard problems and how I solved them
Cost. A long policy packed into every turn was the expensive design. The pipeline embeds once and sends the passages for that question.
Grounding. The system instructions tell the model to answer from the retrieved context. The passages are the input, not an optional hint.
More than one model on eSanad. The chatbot can move between models without a second ingestion pipeline.
Reliability and failure handling
Retrieval is the guard. If the passages are empty, the honest result is that the documents did not contain the answer. Retries belong on the model call, not on a second invented paragraph.
Outcome
The assistants are in the products: document question answering, and the eSanad chatbot on the quote flow. The context design is what keeps the token use tied to the retrieved text.
Stack
Next.js, Python where the pipeline needs it, LangChain, LangGraph, OpenAI, Claude, Pinecone, FAISS, Ollama.
What I would improve next
A fixed set of questions with expected passages, so a retrieval change can be scored instead of sampled by hand.
Client details are confidential. No client data is shown.