sequences

Definition

Ordered data where the order carries meaning — token streams, time series, audio frames, DNA bases. Transformers consume a whole sequence at once, which is what distinguishes them from recurrent models. Because attention is permutation-invariant by construction, order has to be injected explicitly through position embeddings; without them the model would treat the input as a set, and shuffled text would look identical.

Explainers using this term

• Mamba and State Space Models — Handling Sequences Without AttentionEfficiently Modeling Long Sequences with Structured State Spaces