#linear-algebra

6 articles

01 ·Linear Algebra·FREE·PAPER·11 min read PCA from Scratch — Eigenvectors as the Axes of Your Data A cloud of correlated data has a grain — a direction along which it varies most. Principal component analysis is the machinery that finds that grain. This article builds it from one picture: center the data, ask which direction carries the most variance, and discover that the answer is an eigenvector of the covariance matrix. Eigenvalues turn out to be the variance along each axis, the top few axes give you dimensionality reduction, and the explained-variance ratio tells you what you threw away. 02 ·Numerical Computing·★ MEMBER·13 min read Solving Systems of Equations — Direct Methods and Iterative Methods How much a bridge sags, how heat spreads through a room, what a Gaussian process predicts — once a computer gets hold of them they all turn into the same shape, Ax = b. This article builds up elimination (LU) and approximation (conjugate gradients) from zero, through why a million-unknown system can't be solved by elimination, all the way to condition numbers, preconditioning, and matrix-free solvers. 03 ·Linear Algebra·★ MEMBER·13 min read A Tour of Matrix Decompositions — When to Reach for LU, QR, Cholesky, or SVD A decomposition rewrites one awkward transformation as a product of parts you can actually work with: solve it (LU), solve it at half price (Cholesky), orthogonalize and fit (QR), or survive a broken matrix (SVD). Starting from why nobody computes an inverse, this piece builds up to a single table that tells you which one to reach for, plus the numpy that goes with it. 04 ·Linear Algebra·FREE·7 min read The Linear Algebra Under LoRA and RAG — Eigenvalues, Low Rank and Vector Search, Hands On A matrix is a deformation of space, an eigenvector is a direction that survives it, SVD generalises the idea, and the dot product is the definition of 'similar'. Four interactive figures and four equations show that LoRA's ΔW=BA and RAG's vector search stand on the same floor. A column meant to be dragged, not just read. 05 ·Linear Algebra·★ MEMBER·PAPER·10 min read Singular Value Decomposition and Low-Rank Approximation — the Math Behind LoRA Starting from the 'rotate, stretch, rotate' picture, this article builds Singular Value Decomposition (SVD) from zero: matrices as stacks of rank-1 layers, why real-world data needs only a few of them, and how that single fact lets LoRA fine-tune a giant model with 0.4% of the parameters. 06 ·Linear Algebra·FREE·9 min read Linear Algebra for AI — What Vectors and Matrices Are Actually Doing You will never compute a determinant to read an AI paper. What you do need is two ideas: a vector is a coordinate where meaning lives, and a matrix is a machine that carries coordinates somewhere else. From why a dot product means similarity to reading the attention equation symbol by symbol.