#state-space-model

1 articles

01 ·Large Language Models·FREE·PAPER·12 min read Mamba and State Space Models — Handling Sequences Without Attention Attention keeps everything and re-reads it on every step, and the price is quadratic cost in sequence length. State space models take the opposite bet: keep one fixed-size note and update it. Starting from a continuous-time linear system, we trace why S4 can secretly become a convolution, what exactly Mamba made 'selective' in order to give that convolution up, and where the whole approach loses to attention.