Efficiency: FlashAttention, quantization, speculative decoding
A · 10-second intuition Make each token cheaper by moving fewer bytes (fused kernels, smaller numbers) or by guessing several tokens and verifying them at once.
Predict before you read
Does FlashAttention change attention's output?