RAG Prompting Patterns

RAG gives the model fresh, private, or authoritative context at query time — but retrieval only helps if the prompt makes the model actually use it. The core instruction is grounding: tell the model to answer using the provided context and to treat its own training knowledge as secondary or off-limits for facts. Without that, it blends retrieved text with half-remembered training data, and you can't tell which is which. A grounded prompt reads like: answer using only the sources below, and if they don't contain the answer, say you don't know. That one constraint is the backbone of every reliable retrieval system you will build.

Structure the context so the model can cite it. Wrap each retrieved chunk in a labeled block with a stable id and, ideally, a source title or URL. Then require inline citations in the output: every claim must cite the source id it came from, like a bracketed doc 2. Citations do triple duty — they let users verify, they let you evaluate retrieval quality by checking whether the cited chunks are the right ones, and they discourage the model from asserting things no source supports. Pick a format and enforce it in your output contract; bracketed ids, footnotes, and inline URLs all work, and consistency is what matters most.

The no-answer case is where RAG systems earn trust or lose it. Retrieval will sometimes return nothing relevant, and the model's instinct is to answer anyway from training data or by stretching a weak chunk. Explicitly authorize and require abstention: if the sources don't answer the question, reply with an exact scripted line like 'I don't have that information in the provided sources.' A scripted out makes refusing feel like following instructions rather than failing. Then handle that string in your app — offer to escalate, search wider, or ask a clarifying question. A confident wrong answer is far more expensive than an honest gap.

Two failure modes to design against. First, conflicting sources: when chunks disagree, tell the model to surface the conflict and cite both rather than silently picking one. Second, stale or irrelevant retrieval poisoning the answer — keep chunks tightly scoped, and consider having the model judge each chunk's relevance before answering, a lightweight in-prompt rerank. And remember that retrieved content is untrusted input: a document could carry injected instructions, so the delimiting-and-labeling discipline from Module 6 applies to every chunk you paste in. Grounding is not just an accuracy tactic; it is also a security boundary.