Probability and softmax
Softmax turns any list of scores into probabilities: all positive, adding up to 1, with bigger scores getting more.
Play with it
Move the scores (logits) of the five candidate words, and the temperature. The bars are the probabilities.
p(i) = e^(zᵢ/T) / Σⱼ e^(zⱼ/T) · total = 100%
Adding the same amount to every logit leaves the bars exactly where they were.
The idea
A model's raw outputs, logits, can be any real numbers. Softmax computes e^score for each and divides by the sum, which makes every value positive and the total exactly 1.
Adding the same number to every logit changes nothing: only differences matter. A gap of 1 between two logits means one is e ≈ 2.72 times as likely as the other.
Temperature divides the logits before softmax. Below 1 the distribution sharpens toward the top choice; above 1 it flattens toward uniform. Temperature 0 means always take the top one (greedy).
Softmax does not choose a token. It only produces probabilities; greedy decoding, sampling, top-k and top-p are separate rules applied afterwards.
Check yourself
Logits [2, 2, 2]: what are the probabilities?
1/3 each. Equal scores give equal probabilities, whatever the value.
What does raising the temperature do?
Flattens the distribution: less likely tokens get picked more often.