Non-determinism

Non-determinism means the same prompt does not reliably give the same answer. The main cause is sampling: next-token prediction produces a probability distribution, and the model picks from it, so two runs can diverge at the first uncertain token and never reconverge. Even at temperature zero, floating-point arithmetic on batched GPU requests introduces small differences, and some architectures route tokens through different internal experts depending on what else is in the batch.

There's a second, sneakier source: the thing you're calling changes under you. Providers update models, harnesses revise their system prompt, and a tool gets a new description. 'It worked yesterday' is a weak claim in agentic coding, and the instinct to find the one prompt that always works is chasing something that doesn't exist.

Design for variance instead of fighting it. Put an automated check behind every agent change so a bad roll fails loudly. Rerun rather than re-argue when a result looks like a fluke. Pin exact model versions in API code. And write your instructions so the intended path is overwhelmingly the most likely one: clear constraints narrow the distribution far more than pleading does.