moe

Definition

Mixture of Experts: a layer holding many parallel expert networks, where a router selects only a few per token, so total parameter count grows while the compute spent on each token stays roughly fixed. This is why MoE sizes must be read twice — total and active parameters differ by a large factor, and only the second predicts inference cost. The hard parts are routing stability, load balancing, and holding all experts in memory while most sit idle.

Explainers using this term

• Paper Walkthrough — SMELT: Is Looping the Same Layers Twice Actually a Win When the Budget Is Matched?SMELT: Scaling Laws for Compute-Matched MoE Looped Transformers
• Paper Explained: FreeToken — Treating Your Own PC as a Single Elastic Inference PlatformFreeToken: Efficient Edge-Native MoE Serving with Bandwidth-Adaptive Execution