Home › Lab
Interactive Lab
Grab self-attention with your hands
Words sit around a round table and attention flies between them as light. Turn off the √d_k division and watch the distribution collapse; flip the causal mask and watch the future go dark.
An honest note about this visualisation
The math is exactly the scaled dot-product attention from the paper. Only the embeddings are toys: an 8-dimensional space designed by hand so the patterns are legible — not weights from a trained model. arXiv:1706.03762 — https://arxiv.org/abs/1706.03762How to play
- Click a card → make it the query (beams fly from it)
- Drag → orbit the table
- Change the head → a different projection, a different focus
- Uncheck “divide by √d_k” → softmax collapses onto one token (footnote 1 of the paper)