gqa

Definition

Grouped-query attention: query heads are partitioned into groups, and all heads within a group share one key/value projection. This shrinks the KV cache that must be held during autoregressive decoding, which is the dominant memory cost at long context lengths. It sits between multi-head attention, where every head keeps its own K and V, and multi-query attention, where all heads share a single pair, and it recovers most of the quality MQA gives up.