#math
48 articles
01
·Distillation & Compression·FREE·PAPER·8 min read
The Math of Distillation — Why Soft Answers Teach More
Why is the distillation loss KL(teacher||student), what is the temperature T actually doing, and where does that mysterious T² factor in every implementation come from? A ground-up derivation of why a soft distribution carries more teaching signal than a correct answer.
02
·Probability & Statistics·★ MEMBER·PAPER·11 min read
Statistical Learning Theory — Why Does Learning Generalize?
You only ever see finitely many examples, yet the model answers questions about data it has never seen. This piece builds the guarantee from scratch — Hoeffding, uniform convergence, VC dimension, PAC — then shows where it broke against deep learning, and what survives in day-to-day evaluation design.
03
·Probability & Statistics·★ MEMBER·PAPER·10 min read
Optimal Transport — The Mathematics of Moving Distributions
Optimal transport measures the distance between two probability distributions as the cheapest way to haul sand from one pile into the shape of another. From the definition of the Wasserstein distance to the Sinkhorn algorithm that makes it practical, and on to WGAN, FID and Flow Matching — with numpy code and the failure modes that bite in production.
04
·Probability & Statistics·★ MEMBER·10 min read
Monte Carlo Methods from Scratch — Solving Integrals with Dice
An integral you cannot solve can still be estimated: scatter random points and take the average. This piece builds up why the law of large numbers buys you only 1/√N, what importance sampling rescues, and how MCMC samples from a distribution whose normalizing constant nobody can compute.
05
·Probability & Statistics·★ MEMBER·10 min read
Monte Carlo Methods from Scratch — Solving Integrals with Dice
An integral you cannot solve can still be estimated: scatter random points and take the average. This piece builds up why the law of large numbers buys you only 1/√N, what importance sampling rescues, and how MCMC samples from a distribution whose normalizing constant nobody can compute.
06
·Probability & Statistics·★ MEMBER·PAPER·11 min read
Kernel Methods and Gaussian Processes — The Champions Before Neural Nets
Before deep learning took over, kernel SVMs held the crown in classification. This article builds the whole story from zero: the kernel trick that makes lifting into high dimensions free, Gaussian processes that put a probability distribution over functions, and Bayesian optimization that turns uncertainty into a weapon.
07
·Calculus & Optimization·★ MEMBER·11 min read
Jacobians and Hessians — Multivariable Calculus, Drawn
The Jacobian is a magnifying glass at a point; the Hessian is how sharply the ground curves there. From local linearization to what eigenvalues say about the terrain, why Newton's method is fast on paper but absent in practice, and how to get Hessian information in a billion dimensions without ever building the matrix.
08
·Calculus & Optimization·★ MEMBER·11 min read
Jacobians and Hessians — Multivariable Calculus, Drawn
The Jacobian is a magnifying glass at a point; the Hessian is how sharply the ground curves there. From local linearization to what eigenvalues say about the terrain, why Newton's method is fast on paper but absent in practice, and how to get Hessian information in a billion dimensions without ever building the matrix.
09
·Linear Algebra·★ MEMBER·PAPER·10 min read
Symmetry and Equivariance — How Group Theory Shapes Network Design
How to bake "a shifted cat is still a cat" into the architecture itself — from the four axioms of a group, through the definition of equivariance, to why convolution is the only translation-equivariant linear map and how AlphaFold handles 3D geometry. Including the cases where imposing symmetry backfires.
10
·Linear Algebra·★ MEMBER·PAPER·10 min read
Symmetry and Equivariance — How Group Theory Shapes Network Design
How to bake "a shifted cat is still a cat" into the architecture itself — from the four axioms of a group, through the definition of equivariance, to why convolution is the only translation-equivariant linear map and how AlphaFold handles 3D geometry. Including the cases where imposing symmetry backfires.
11
·Information Theory·★ MEMBER·PAPER·11 min read
Compression Is Prediction Is Intelligence — LLMs Through Information Theory
Training a model to guess the next token is training it to shrink a file. Route the prediction through arithmetic coding and "probability model" and "code" turn out to be the same object — which makes cross-entropy loss literally the size of the compressed output. Built up from zero, ending at why the Hutter Prize treats compression ratio as an intelligence test.
12
·Calculus & Optimization·★ MEMBER·13 min read
Calculus of Variations — What It Means to Differentiate a Function
When the thing you are searching for is a shape rather than a number, differentiation has to be reinvented. Functionals and the first variation, the Euler–Lagrange equation, why the fastest slide is a cycloid, and how all of it lands on the ELBO — the variational problem machine learning runs most.
13
·Calculus & Optimization·★ MEMBER·13 min read
Calculus of Variations — What It Means to Differentiate a Function
When the thing you are searching for is a shape rather than a number, differentiation has to be reinvented. Functionals and the first variation, the Euler–Lagrange equation, why the fastest slide is a cycloid, and how all of it lands on the ELBO — the variational problem machine learning runs most.
14
·Probability & Statistics·★ MEMBER·10 min read
Hypothesis Testing and A/B Tests — How to Use a p-value, and How People Misuse It
Ship the identical page to two groups and the numbers still won't match. A hypothesis test converts that gap into 'how many units of noise is this?' — that number is the p-value. From the null hypothesis up through the three failures that actually ruin experiments: power, peeking, and multiple comparisons.
15
·Probability & Statistics·★ MEMBER·10 min read
Hypothesis Testing and A/B Tests — How to Use a p-value, and How People Misuse It
Ship the identical page to two groups and the numbers still won't match. A hypothesis test converts that gap into 'how many units of noise is this?' — that number is the p-value. From the null hypothesis up through the three failures that actually ruin experiments: power, peeking, and multiple comparisons.
16
·Linear Algebra·FREE·10 min read
Tensors and Shape Manipulation — If You Can Read einsum, You Can Read Papers
The Σ_j A_ij B_jk in the paper and the x.transpose(1,2) in the code say the same thing, and einsum is the bridge between them. Three tools — axes, broadcasting, contraction — are enough to write attention in a single line.
17
·Linear Algebra·FREE·10 min read
Tensors and Shape Manipulation — If You Can Read einsum, You Can Read Papers
The Σ_j A_ij B_jk in the paper and the x.transpose(1,2) in the code say the same thing, and einsum is the bridge between them. Three tools — axes, broadcasting, contraction — are enough to write attention in a single line.
18
·Probability & Statistics·FREE·13 min read
A Field Guide to Probability Distributions — Where Normal, Poisson, and the Exponential Family Come From
The normal and Poisson distributions aren't formulas to memorize — they're the shapes that certain situations inevitably produce. Follow three stories (adding things up, counting rare events, and refusing to assume more than you know) and the textbook zoo collapses into one river called the exponential family, with sigmoid and softmax falling out of it.
19
·Probability & Statistics·FREE·13 min read
A Field Guide to Probability Distributions — Where Normal, Poisson, and the Exponential Family Come From
The normal and Poisson distributions aren't formulas to memorize — they're the shapes that certain situations inevitably produce. Follow three stories (adding things up, counting rare events, and refusing to assume more than you know) and the textbook zoo collapses into one river called the exponential family, with sigmoid and softmax falling out of it.
20
·Information Theory·★ MEMBER·PAPER·11 min read
Mutual Information — Putting a Number on What You Know
How much does knowing one thing shrink your uncertainty about another? Mutual information turns that into a single number. We build it from entropy subtraction, work through conditional MI, the data processing inequality and why estimation is hard, and end at why the InfoNCE loss behind modern contrastive learning is called a lower bound on mutual information.
21
·Information Theory·★ MEMBER·PAPER·11 min read
Mutual Information — Putting a Number on What You Know
How much does knowing one thing shrink your uncertainty about another? Mutual information turns that into a single number. We build it from entropy subtraction, work through conditional MI, the data processing inequality and why estimation is hard, and end at why the InfoNCE loss behind modern contrastive learning is called a lower bound on mutual information.
22
·Calculus & Optimization·★ MEMBER·11 min read
Matrix Calculus from Scratch — Derive the Backward Pass Yourself
Where does the transpose in ∂L/∂W = XᵀG actually come from? Matrix calculus is not a formula sheet to memorize — it is one move: rotate dX to the right inside a trace. From denominator layout and shape-checking to the gradients of a linear layer and softmax + cross-entropy, ending with a double-precision gradient check.
23
·Calculus & Optimization·★ MEMBER·11 min read
Matrix Calculus from Scratch — Derive the Backward Pass Yourself
Where does the transpose in ∂L/∂W = XᵀG actually come from? Matrix calculus is not a formula sheet to memorize — it is one move: rotate dX to the right inside a trace. From denominator layout and shape-checking to the gradients of a linear layer and softmax + cross-entropy, ending with a double-precision gradient check.
24
·Calculus & Optimization·★ MEMBER·PAPER·12 min read
Beyond SGD — Adam, Second-Order Methods, and Constrained Optimization
What exactly is momentum accumulating? What does each of Adam's four lines do? What did AdamW fix? And why does nobody train an LLM with second-order methods that are supposedly faster? Metaphor, equations, live figures, code, and production practice — no prerequisites assumed.
25
·Calculus & Optimization·★ MEMBER·PAPER·12 min read
Beyond SGD — Adam, Second-Order Methods, and Constrained Optimization
What exactly is momentum accumulating? What does each of Adam's four lines do? What did AdamW fix? And why does nobody train an LLM with second-order methods that are supposedly faster? Metaphor, equations, live figures, code, and production practice — no prerequisites assumed.
26
·Linear Algebra·★ MEMBER·13 min read
A Tour of Matrix Decompositions — When to Reach for LU, QR, Cholesky, or SVD
A decomposition rewrites one awkward transformation as a product of parts you can actually work with: solve it (LU), solve it at half price (Cholesky), orthogonalize and fit (QR), or survive a broken matrix (SVD). Starting from why nobody computes an inverse, this piece builds up to a single table that tells you which one to reach for, plus the numpy that goes with it.
27
·Linear Algebra·★ MEMBER·13 min read
A Tour of Matrix Decompositions — When to Reach for LU, QR, Cholesky, or SVD
A decomposition rewrites one awkward transformation as a product of parts you can actually work with: solve it (LU), solve it at half price (Cholesky), orthogonalize and fit (QR), or survive a broken matrix (SVD). Starting from why nobody computes an inverse, this piece builds up to a single table that tells you which one to reach for, plus the numpy that goes with it.
28
·Probability & Statistics·★ MEMBER·11 min read
Markov Chains from Scratch — The Process That Only Looks at Now
What happens next depends only on where you are now — that single act of forgetting is a Markov chain. From transition matrices and stationary distributions to why PageRank is an eigenvector and why MCMC gets to ignore the normalizing constant, built from nothing assumed.
29
·Probability & Statistics·★ MEMBER·11 min read
Markov Chains from Scratch — The Process That Only Looks at Now
What happens next depends only on where you are now — that single act of forgetting is a Markov chain. From transition matrices and stationary distributions to why PageRank is an eigenvector and why MCMC gets to ignore the normalizing constant, built from nothing assumed.
30
·Information Theory·FREE·9 min read
Entropy and Cross-Entropy — Where the Loss Function Comes From
The cross-entropy loss you write in every classifier wasn't handed down by decree. Starting from the idea of surprise, we walk a single path through entropy and cross-entropy to that familiar one-liner — and finish with why squared error is the wrong tool for the job.
31
·Information Theory·FREE·9 min read
Entropy and Cross-Entropy — Where the Loss Function Comes From
The cross-entropy loss you write in every classifier wasn't handed down by decree. Starting from the idea of surprise, we walk a single path through entropy and cross-entropy to that familiar one-liner — and finish with why squared error is the wrong tool for the job.
32
·Probability & Statistics·★ MEMBER·11 min read
Thinking Bayesian — A Working Feel for Priors, Likelihoods, and Posteriors
Bayesian updating is the act of feeding yesterday's posterior back in as today's prior. With a conjugate prior the whole update collapses into adding pseudo-counts, and an A/B test becomes two numbers: the probability of winning and the expected loss. From zero background to the traps in stopping rules and prior choice.
33
·Linear Algebra·FREE·7 min read
The Linear Algebra Under LoRA and RAG — Eigenvalues, Low Rank and Vector Search, Hands On
A matrix is a deformation of space, an eigenvector is a direction that survives it, SVD generalises the idea, and the dot product is the definition of 'similar'. Four interactive figures and four equations show that LoRA's ΔW=BA and RAG's vector search stand on the same floor. A column meant to be dragged, not just read.
34
·Linear Algebra·FREE·7 min read
The Linear Algebra Under LoRA and RAG — Eigenvalues, Low Rank and Vector Search, Hands On
A matrix is a deformation of space, an eigenvector is a direction that survives it, SVD generalises the idea, and the dot product is the definition of 'similar'. Four interactive figures and four equations show that LoRA's ΔW=BA and RAG's vector search stand on the same floor. A column meant to be dragged, not just read.
35
·Linear Algebra·★ MEMBER·PAPER·10 min read
Singular Value Decomposition and Low-Rank Approximation — the Math Behind LoRA
Starting from the 'rotate, stretch, rotate' picture, this article builds Singular Value Decomposition (SVD) from zero: matrices as stacks of rank-1 layers, why real-world data needs only a few of them, and how that single fact lets LoRA fine-tune a giant model with 0.4% of the parameters.
36
·Linear Algebra·★ MEMBER·PAPER·10 min read
Singular Value Decomposition and Low-Rank Approximation — the Math Behind LoRA
Starting from the 'rotate, stretch, rotate' picture, this article builds Singular Value Decomposition (SVD) from zero: matrices as stacks of rank-1 layers, why real-world data needs only a few of them, and how that single fact lets LoRA fine-tune a giant model with 0.4% of the parameters.
37
·Information Theory·★ MEMBER·PAPER·9 min read
KL Divergence From Scratch — Measuring the Gap Between Two Distributions
KL divergence measures the gap between two probability distributions. We build it up from a compression metaphor to the definition, its famous asymmetry, and a numpy implementation — then watch it at work as the regularizer in VAEs and the leash in RLHF.
38
·Calculus & Optimization·★ MEMBER·PAPER·9 min read
Convexity and Optimization — Why Deep Learning Works Even Though It Isn't Convex
Optimization textbooks teach a stark divide: convex problems are solvable, non-convex ones come with no guarantees. So why does deep learning — whose loss surface is provably non-convex — work at all? From convex sets and functions to saddle points and flat minima, this article connects the whole story in the language of landscapes.
39
·Calculus & Optimization·★ MEMBER·PAPER·9 min read
Convexity and Optimization — Why Deep Learning Works Even Though It Isn't Convex
Optimization textbooks teach a stark divide: convex problems are solvable, non-convex ones come with no guarantees. So why does deep learning — whose loss surface is provably non-convex — work at all? From convex sets and functions to saddle points and flat minima, this article connects the whole story in the language of landscapes.
40
·Probability & Statistics·★ MEMBER·PAPER·10 min read
Bayes' Theorem in AI — Priors, Posteriors, and Uncertainty
One line of math — Bayes' theorem — turns into three workhorse tools in real AI systems: probability calibration, active learning, and Bayesian optimization. Starting from a positive medical test, we build up priors, posteriors, and uncertainty with zero background assumed.
41
·Probability & Statistics·★ MEMBER·8 min read
Probability and Statistics for AI — A Model's Output Is a Distribution
Classifiers and language models do not return answers; they return probability distributions. Distributions, expectation, conditional probability and Bayes explained from the symbols up — building to the payoff: why maximum likelihood is where loss functions come from. Cross-entropy and MSE were derived, not invented.
42
·Probability & Statistics·★ MEMBER·8 min read
Probability and Statistics for AI — A Model's Output Is a Distribution
Classifiers and language models do not return answers; they return probability distributions. Distributions, expectation, conditional probability and Bayes explained from the symbols up — building to the payoff: why maximum likelihood is where loss functions come from. Cross-entropy and MSE were derived, not invented.
43
·Linear Algebra·FREE·9 min read
Linear Algebra for AI — What Vectors and Matrices Are Actually Doing
You will never compute a determinant to read an AI paper. What you do need is two ideas: a vector is a coordinate where meaning lives, and a matrix is a machine that carries coordinates somewhere else. From why a dot product means similarity to reading the attention equation symbol by symbol.
44
·Linear Algebra·FREE·9 min read
Linear Algebra for AI — What Vectors and Matrices Are Actually Doing
You will never compute a determinant to read an AI paper. What you do need is two ideas: a vector is a coordinate where meaning lives, and a matrix is a machine that carries coordinates somewhere else. From why a dot product means similarity to reading the attention equation symbol by symbol.
45
·Information Theory·★ MEMBER·9 min read
Information Theory and AI — Where Cross-Entropy Loss Came From
Information is surprise, entropy is average surprise, KL divergence is the gap between two distributions. Stack those three and the loss function you already use for classification falls out as the only reasonable answer — including why it involves a logarithm, and what perplexity is really counting.
46
·Information Theory·★ MEMBER·9 min read
Information Theory and AI — Where Cross-Entropy Loss Came From
Information is surprise, entropy is average surprise, KL divergence is the gap between two distributions. Stack those three and the loss function you already use for classification falls out as the only reasonable answer — including why it involves a logarithm, and what perplexity is really counting.
47
·Calculus & Optimization·FREE·7 min read
Calculus for AI — The Gradient Is an Arrow Saying Which Way Is Better
No epsilon-delta limits, no integration by parts. Training is measuring a slope and stepping the other way. A derivative is a multiplier, a gradient is a list of slopes, the chain rule is multiplication — and Jacobians and Hessians only need to be recognised, not computed.
48
·Calculus & Optimization·FREE·7 min read
Calculus for AI — The Gradient Is an Arrow Saying Which Way Is Better
No epsilon-delta limits, no integration by parts. Training is measuring a slope and stepping the other way. A derivative is a multiplier, a gradient is a list of slopes, the chain rule is multiplication — and Jacobians and Hessians only need to be recognised, not computed.