Causal attention, end to end
A · 10-second intuition Each position mixes in information from itself and earlier positions, weighted by learned relevance, and never from the future.
Predict before you read
What does the causal mask do in training?
A · 10-second intuition Each position mixes in information from itself and earlier positions, weighted by learned relevance, and never from the future.
What does the causal mask do in training?