encoder

Definition

The part that maps an input into an internal representation. In a Transformer the encoder attends bidirectionally, so every position sees the whole sequence — the right shape for understanding tasks, which is why BERT-family models are encoder-only. Its counterpart is the decoder, which generates one step at a time under a causal mask; tasks needing both comprehension and generation, such as translation, use the encoder–decoder arrangement.