RAG Fundamentals
Embeddings, chunking, vector search — how retrieval anchors the model in your truth instead of its training data.
RAG (retrieval-augmented generation) answers from YOUR documents: at query time you retrieve the most relevant passages and put them in context with instructions to answer from them and cite. It exists because model weights are frozen, private data was never in them, and un-grounded models fill gaps confidently. RAG is how you get current, private, attributable answers — and 'answer only from the provided context; say so if it's not there' is the cheapest hallucination reducer in the field.
The ingestion pipeline: split documents into chunks, embed each chunk into a vector (embeddings put semantically similar text near each other geometrically), store vectors in an index (pgvector, Qdrant, Pinecone, Chroma — pick by ops preference; pgvector wins when you already run Postgres). At query time: embed the question, nearest-neighbor search, take top-k. Chunking is the underrated knob: chunks must be self-contained enough to make sense alone yet small enough to be precise. Splitting on structure (headings, paragraphs) with modest overlap beats fixed character counts; attach metadata (title, section, date) to every chunk for filtering and citation.
Know when NOT to RAG. Fits-in-context beats RAG for small corpora — with big windows and prompt caching, stuffing a few hundred pages in directly is simpler and often better. RAG earns its complexity when corpora are large, fresh, per-user, or need citations. And retrieval quality ceilings everything: if the right passage isn't in the top-k, no prompt can save the answer — which is why Module 5's eval habits start with measuring retrieval itself.