Computer Architecture
Logic, clocking, the memory hierarchy and the memory wall
01
·Computer Architecture·★ MEMBER·9 min read
The Memory Wall from Scratch — Why Moving Data Costs More Than Computing
Multiplying two numbers is cheap; delivering them is not. Starting from the physics of charging a wire, we get to why DRAM latency never shrank, the orders of magnitude in the memory hierarchy, Little's law, machine balance and the roofline — and end with a procedure for deciding whether your kernel is compute bound or bandwidth bound.
02
·Computer Architecture·★ MEMBER·PAPER·9 min read
The GPU Memory Hierarchy — HBM, SRAM, Registers, and Why Movement Wins
What sets a GPU's speed is not the arithmetic units but where the data sits — registers, shared memory, L2 or HBM — and how many times it is moved. Capacities and bandwidths by order of magnitude, arithmetic intensity and tiling, a roofline per level of the hierarchy, and finally FlashAttention: more FLOPs, less time.
03
·Computer Architecture·★ MEMBER·PAPER·10 min read
CPU Pipelines and Branch Prediction — The Factory Inside One Clock Tick
An instruction appears to finish in a single clock tick because the machine is an assembly line. From the five classic stages, through the three kinds of hazard, to branch prediction, speculative execution, and the day the leftovers of speculation turned into an information leak — Spectre.
04
·Computer Architecture·★ MEMBER·PAPER·10 min read
Systolic Arrays — Building the Heart of the TPU From Scratch
Piling on multipliers doesn't make a chip faster, because moving data costs more than the arithmetic does. This piece takes the grid at the center of the TPU — the systolic array — and shows exactly how it maps the triple loop of matrix multiplication onto rows, columns and clock ticks, from a 2×2 hand trace to a cycle-accurate simulator to the batch-size and shape rules you'll actually tune.
05
·Computer Architecture·★ MEMBER·9 min read
Interconnects — How NVLink, PCIe, and Light Set the Limits of Scale
When adding GPUs stops making things faster, the wiring is usually to blame. This piece pins down the bandwidth hierarchy from HBM through NVLink, PCIe, and the inter-node fabric, breaks collective time into 'steps × latency + bytes ÷ bandwidth', and explains why all-to-all hurts and why distance eventually demands light.