What temperature actually does to a softmax

I’d always read temperature as “makes the model more random”, which is true but not useful. It’s a divisor applied to the logits, before the softmax normalises them:

$$p_i = \frac{\exp(z_i / T)}{\sum_j \exp(z_j / T)}$$

The key consequence: because $\exp$ is applied after the division, scaling the logits does not scale the probabilities proportionally. It changes the ratio between them. For two tokens, the odds ratio is

$$\frac{p_i}{p_j} = \exp\!\left(\frac{z_i - z_j}{T}\right)$$

so the gap $z_i - z_j$ gets amplified as $T \to 0$ and flattened as $T \to \infty$.

• $T \to 0$: the ratio blows up, all mass collapses onto the argmax — greedy decoding.
• $T = 1$: the distribution the model was trained to produce.
• $T \to \infty$: every exponent goes to $0$, so $p_i \to 1/n$ — uniform.

The thing I’d been getting wrong: temperature is not a post-hoc reweighting of a fixed distribution. Applying it to probabilities rather than logits gives a genuinely different result.

Where I learned it

Re-derived it from the softmax definition after being unable to explain to someone why temperature=0 is deterministic.