transformers

Definition

The family of architectures built around self-attention, introduced for machine translation and since adopted far beyond text — vision, audio, protein sequences, robotics. Their appeal is parallel training over long contexts without recurrence. Note the collision: transformers also names the widely used Hugging Face library, so in code-facing papers the word may refer to the package rather than the architecture.

Explainers using this term

• Paper Walkthrough — SMELT: Is Looping the Same Layers Twice Actually a Win When the Budget Is Matched?SMELT: Scaling Laws for Compute-Matched MoE Looped Transformers
• Encoder or Decoder — The Fork in the Road Between BERT and GPTBERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
• Do Transformers Actually Work on Time Series? — The Argument and the Practical AnswerAre Transformers Effective for Time Series Forecasting?