#memory-hierarchy

1 articles

01 ·Computer Architecture·FREE·PAPER·9 min read The GPU Memory Hierarchy — HBM, SRAM, Registers, and Why Movement Wins What sets a GPU's speed is not the arithmetic units but where the data sits — registers, shared memory, L2 or HBM — and how many times it is moved. Capacities and bandwidths by order of magnitude, arithmetic intensity and tiling, a roofline per level of the hierarchy, and finally FlashAttention: more FLOPs, less time.