MHA, MQA and GQA
A · 10-second intuition Grouped-query attention lets several query heads share one key/value head, cutting the memory that generation needs.
Predict before you read
If Qwen3 used MHA instead of GQA, how would its KV cache change?
Not started