kernel

Definition

A unit of computation launched on an accelerator — one matmul, one softmax, one normalization. Performance work in this field is largely kernel work: fusing several operations into a single kernel so intermediate tensors never round-trip through high-bandwidth memory, as FlashAttention does for attention. Note the collision with the other machine-learning sense of kernel, as in kernel methods and the kernel trick; context makes clear which one is meant.