Long-Context Prompting
Placement, labeling, and quote-then-answer — how to prompt reliably when the context is huge and attention isn't uniform.
Long context is a budget you spend, not a filing cabinet you fill. Frontier models in 2026 carry hundreds of thousands to over a million tokens, but attention across them is not uniform. Models attend most reliably to the very start and very end of the window and get soft in the middle — the 'lost in the middle' effect, well-documented and still real on the newest models. So placement is a lever: put the question and the most important instructions at the very end, closest to generation, and stable framing at the top. The single worst place for the fact you need is the exact middle of a huge dump.
Order documents deliberately and label every one. When you paste multiple sources, wrap each in a tag with an id and title, like a doc block marked id 3, title Q3 earnings, so the model can reference them precisely and you can trace where an answer came from. Put the most relevant documents last when you can rank them by recency or retrieval score. Unlabeled concatenation forces the model to guess boundaries; labeled chunks let it cite 'doc 3' and let you verify. This also makes the citation formats from the RAG lesson trivial to enforce, because every claim can point at a real, named source.
The highest-leverage long-context technique is quote-first, then answer. Instruct the model to extract the exact verbatim sentences relevant to the question before it writes anything. This forces it to actually locate evidence in the haystack rather than pattern-matching from memory, and it gives you a checkable trail: if the quotes are wrong or missing, you distrust the answer. It costs output tokens but slashes hallucination on large documents. Pair it with 'if the answer isn't in the documents, say so' to stop confident fabrication when the relevant passage simply isn't present.
Even with a big window, more context is not free or always better. Every irrelevant token dilutes attention and adds cost and latency, and past a point accuracy drops — stuffing the whole wiki when three sections would do makes answers worse, not better. Treat retrieval and summarization as ways to raise signal density, not fallbacks for small windows. And exploit prompt caching: put the large stable corpus first so it is cached across calls, and vary only the question at the end. Long context and good retrieval are partners, not rivals — the window is the desk, retrieval decides what lands on it.