Autoregressive language modeling
A · 10-second intuition A language model is a function that turns all the text so far into a probability for every possible next token.
Predict before you read
During training on a 1,000-token document, how many next-token predictions are scored?