Course contents
Temperature, Top-p and Why Outputs Vary
See how sampling turns probabilities into words, why most frontier APIs no longer let you set temperature, and where consistency and variety come from instead.
Run the same prompt twice and you can get two different answers. That isn't a malfunction. At every step the model assigns a probability to each possible next token, and the API samples one of them. Two classic dials, temperature and top-p, change how that sample is drawn. Try them before reading on.
A toy distribution over six words. A real model weighs tens of thousands of tokens at every step.
Move the controls and watch what changes.
Drag temperature to 0 and sample several times. What happens?
What the two dials do
Temperature reshapes the whole distribution. Below 1 it sharpens toward the favorites; above 1 it flattens toward the long tail. Top-p, also called nucleus sampling, keeps the smallest set of tokens whose probabilities add up to p and drops the rest. Temperature changes the odds; top-p decides which words are allowed in at all.
Set temperature back to 1 and top-p to 0.50. What does the widget show?
In 2026 you mostly can't turn them
Reasoning models are tuned to work at their default sampling, and most vendors now lock it. Google's docs say to keep these values at their defaults on earlier Gemini 3 models “to avoid degraded reasoning performance,” and xAI says Grok models “don't respond well to other sampling settings.” As of November 2026:
| Model | Can you set temperature and top-p? |
|---|---|
| Claude Fable 5.2, Opus 5.5, Sonnet 5.5, Haiku 5.5 | No. Non-default values return a 400 error, and from API version 2026-10-21 sending them at all does. |
| GPT-6 Astra and GPT-6.1 Sol | No. OpenAI supports them only at reasoning effort none, which these models don't offer. |
| GPT-6.1 Terra and GPT-6 Luna | Only with reasoning effort set to none. |
| Gemini 4 Carbon and Argon | You can send them, but Gemini 4 ignores custom values. |
| Grok 4.7 | Yes, but xAI recommends temperature 0.7 and top-p 0.95 and says quality usually drops if you stray. |
| Open-weight models you run yourself | Yes, fully. This is where the widget's dials still apply directly. |
temperature: 0 fails outright on Claude's current models. Remove the parameter, and move the job it was doing into the prompt and the output contract.Where consistency comes from now
If you used temperature 0 to make outputs repeatable, the lever has moved into the spec. Five things make answers stable: a precise task, a closed set of allowed answers, examples of the hard cases, an output contract enforced with structured outputs, and source documents for anything factual. Even where temperature 0 is still allowed, hosted APIs don't promise identical output on every run, so test stability by running each important case several times.
Where variety comes from now
If you used a high temperature for creativity, ask for the variety directly. Request several options in one call and name the dimension they should differ on. Anthropic's Sonnet 5 guide recommends this for design work: because temperature isn't accepted, have the model “propose 4 distinct visual directions” before it builds one. You can also rotate examples, perspectives or constraints between calls.
On a current frontier reasoning model, does each approach get you what you want?
Setting temperature to 0 so audit labels repeat exactly
The repeatable classifier
You're moving an audit-log classifier to Claude Opus 5.5. Auditors need the same input to get the same label.
The first test request fails with a 400 error. The old code sends
temperature: 0.
Plain-English definitions from the AI Coding Dictionary.