encoder

Definition

The part that maps an input into an internal representation. In a Transformer the encoder attends bidirectionally, so every position sees the whole sequence — the right shape for understanding tasks, which is why BERT-family models are encoder-only. Its counterpart is the decoder, which generates one step at a time under a causal mask; tasks needing both comprehension and generation, such as translation, use the encoder–decoder arrangement.

Explainers using this term

• Paper Walkthrough: X-AuT — Pull Audio-Encoder Layers Out Two at a Time, Then Backfill from a Bigger TeacherX-AuT: Progressive Audio-Encoder Compression for Speech LLMs with Cross-Scale Distillation