Inference: prefill, KV cache, decode

A · 10-second intuition Reading the prompt happens all at once (prefill); writing the answer happens one token at a time (decode), reusing stored keys and values.