Linear Algebra
Vectors, matrices, eigenvalues, SVD — what a dimension really is
01
·Linear Algebra·FREE·9 min read
Linear Algebra for AI — What Vectors and Matrices Are Actually Doing
You will never compute a determinant to read an AI paper. What you do need is two ideas: a vector is a coordinate where meaning lives, and a matrix is a machine that carries coordinates somewhere else. From why a dot product means similarity to reading the attention equation symbol by symbol.
02
·Linear Algebra·FREE·7 min read
The Linear Algebra Under LoRA and RAG — Eigenvalues, Low Rank and Vector Search, Hands On
A matrix is a deformation of space, an eigenvector is a direction that survives it, SVD generalises the idea, and the dot product is the definition of 'similar'. Four interactive figures and four equations show that LoRA's ΔW=BA and RAG's vector search stand on the same floor. A column meant to be dragged, not just read.
03
·Linear Algebra·★ MEMBER·PAPER·10 min read
Singular Value Decomposition and Low-Rank Approximation — the Math Behind LoRA
Starting from the 'rotate, stretch, rotate' picture, this article builds Singular Value Decomposition (SVD) from zero: matrices as stacks of rank-1 layers, why real-world data needs only a few of them, and how that single fact lets LoRA fine-tune a giant model with 0.4% of the parameters.
04
·Linear Algebra·FREE·PAPER·11 min read
PCA from Scratch — Eigenvectors as the Axes of Your Data
A cloud of correlated data has a grain — a direction along which it varies most. Principal component analysis is the machinery that finds that grain. This article builds it from one picture: center the data, ask which direction carries the most variance, and discover that the answer is an eigenvector of the covariance matrix. Eigenvalues turn out to be the variance along each axis, the top few axes give you dimensionality reduction, and the explained-variance ratio tells you what you threw away.
05
·Linear Algebra·★ MEMBER·13 min read
A Tour of Matrix Decompositions — When to Reach for LU, QR, Cholesky, or SVD
A decomposition rewrites one awkward transformation as a product of parts you can actually work with: solve it (LU), solve it at half price (Cholesky), orthogonalize and fit (QR), or survive a broken matrix (SVD). Starting from why nobody computes an inverse, this piece builds up to a single table that tells you which one to reach for, plus the numpy that goes with it.
06
·Linear Algebra·FREE·10 min read
Tensors and Shape Manipulation — If You Can Read einsum, You Can Read Papers
The Σ_j A_ij B_jk in the paper and the x.transpose(1,2) in the code say the same thing, and einsum is the bridge between them. Three tools — axes, broadcasting, contraction — are enough to write attention in a single line.
07
·Linear Algebra·★ MEMBER·PAPER·10 min read
Symmetry and Equivariance — How Group Theory Shapes Network Design
How to bake "a shifted cat is still a cat" into the architecture itself — from the four axioms of a group, through the definition of equivariance, to why convolution is the only translation-equivariant linear map and how AlphaFold handles 3D geometry. Including the cases where imposing symmetry backfires.