#bert

1 articles

01 ·How Transformers Work·★ MEMBER·PAPER·9 min read Encoder or Decoder — The Fork in the Road Between BERT and GPT One masked triangle in the attention table is what separated BERT from GPT. This piece works through bidirectional versus autoregressive with the equations and an interactive figure, then asks why generation won and where encoders are still the first choice.