tensor

Definition

A multi-dimensional array of numbers — the container for essentially all data and weights in deep learning, with scalars, vectors, and matrices as low-dimensional special cases. GPUs exist commercially in their current form because bulk operations on tensors parallelize so well. Mathematicians and physicists mean something stricter, involving transformation rules under change of basis; in machine learning the word almost always just means the array.

Explainers using this term

• Systolic Arrays — Building the Heart of the TPU From ScratchIn-Datacenter Performance Analysis of a Tensor Processing Unit
• How AI Accelerators Are Designed — What Actually Separates GPUs, NPUs, and TPUsIn-Datacenter Performance Analysis of a Tensor Processing Unit