Prefix cache

The prefix cache is a model provider feature that stores the processed form of the start of your prompt so the next model provider request can skip re-reading it. Because a coding session re-sends the same system prompt, tool definitions and conversation history on every request, most of each request is identical to the one before. A cache hit turns those input tokens into much cheaper cache tokens and shortens the wait before the first new token.

The catch is the word prefix. Caching works from the start of the prompt up to the first change; anything after that point is processed fresh. Change something early (edit the system prompt, reorder tools, alter an old message) and the whole cache from there onward is lost. On most providers a cache that sits idle for a few minutes is dropped, so a long coffee break can mean the next request re-pays for the full context.

Harnesses like Claude Code and Codex handle this automatically and structure requests so the stable parts come first. Your part is to keep the stable parts stable: don't churn project instructions mid-session, avoid tools that inject changing timestamps at the top of the context, and keep working steadily rather than in bursts separated by long gaps.