Students Professional
JA EN

#learning-rate

1 articles

01 ·Training & Alignment·★ MEMBER·PAPER·10 min read Learning Rate Schedules — Why Warmup and Why Cosine The learning rate is not a fixed number — it is a curve you design across the whole run. Why we deliberately start slow (warmup), why we come down along a cosine, and what else has to move when batch size changes. Equations, live figures, PyTorch code, and the mistakes that actually break runs.