Final norm and LM head
A · 10-second intuition After the last block, the final vector is normalized and compared with one output direction per vocabulary token to produce scores.
Predict before you read
Adding 5 to every logit changes the probabilities how?