transformers

Definition

The family of architectures built around self-attention, introduced for machine translation and since adopted far beyond text — vision, audio, protein sequences, robotics. Their appeal is parallel training over long contexts without recurrence. Note the collision: transformers also names the widely used Hugging Face library, so in code-facing papers the word may refer to the package rather than the architecture.

Explainers using this term

• Paper Deep Dive: Transformers Stop Thinking Too Early — How a One-Layer Rank-8 LoRA Stretches 3-Line Reference Following to 50Transformers Stop Thinking Too Early, and a Tiny LoRA Fixes It
• Paper Walkthrough: Marigold V2 — Turning an Image Editor Into a State-of-the-Art Depth Estimator on One 32GB GPUMarigold V2: Revisiting Diffusion Transformers for Monocular Depth Estimation
• SMELT explained (arXiv:2609.01343) — compute-matched loopingSMELT: Scaling Laws for Compute-Matched MoE Looped Transformers