decoupled

Definition

Separating two things that were previously entangled so each can be controlled independently. The canonical case is decoupled weight decay: applying decay directly to the weights after the update, instead of folding it into the gradient as an L2 term, so that Adam's per-parameter adaptive scaling no longer distorts the regularization strength. That fix is what distinguishes AdamW from Adam, and it made weight decay a tunable knob again.