Students Professional
JA EN

#attention-sink

1 articles

01 ·Inference & Serving·★ MEMBER·PAPER·15 min read Paper Walkthrough — SMELT: Is Looping the Same Layers Twice Actually a Win When the Budget Is Matched? A study of looped Transformers that finally controls the comparison: per-token FLOPs, total parameters, and KV cache are all held close. The resulting recipe, SMELT, loops the middle half twice and reports 6.8–18.0% training-FLOPs savings on the compute-optimal frontier.