exact

Definition

Returning the mathematically defined result rather than an approximation. The word does real work in efficiency papers: exact attention means the output is identical to standard attention, which separates methods like FlashAttention from sparse, low-rank, or linearized attention, where quality can shift in ways that only show up on some inputs. Keep it distinct from accurate — exact is a property of the computation, accurate is a property of the answer.

Explainers using this term

• FlashAttention from Scratch — The Paradox of Doing More Math to Go FasterFlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness
• The GPU Memory Hierarchy — HBM, SRAM, Registers, and Why Movement WinsFlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness