#deep-learning
12 articles
01
·CNNs & Image Recognition·FREE·PAPER·9 min read
The ImageNet Moment — The Day Deep Learning Won
In 2012 an image-recognition contest saw its error rate fall from around 26% to 15% in a single year. Neural networks had existed for three decades — so why then? This is the story of the moment data, compute, and method finally lined up, told down to the technical details.
02
·Numerical Computing·★ MEMBER·12 min read
Build Your Own Autograd — A Mini PyTorch in 100 Lines
Start from a single Value class, add operator overloading, topological ordering, and gradient accumulation, then put a neural network on top and train it. Once you have seen the reasons behind each design choice, zero_grad() and retain_graph stop being trivia to memorize.
03
·Numerical Computing·FREE·10 min read
How Autodiff Actually Works — Unpacking the PyTorch Magic
Why does writing loss.backward() hand you derivatives for millions of parameters? We build up computation graphs, the chain rule, and forward vs. reverse mode from zero — then write a working 40-line autograd engine.
04
·Deep Learning Basics·FREE·PAPER·13 min read
Activation Functions from Scratch — Why Nonlinearity Is Non-Negotiable
Without an activation function, a hundred stacked layers can do exactly what one layer does. Starting from that one-line proof, this article traces why sigmoid was abandoned, why ReLU won, and why today's LLMs settled on SiLU and SwiGLU — with an interactive plot where you can drag the input and watch the slope.
05
·Generative Models·FREE·10 min read
VAEs from Scratch — Stir Probability into "Compress and Restore" and You Get a Generator
An autoencoder that only compresses and restores cannot invent anything new. This walks through why a single drop of probability turns it into a generative model — ELBO, the reparameterization trick, and walking the latent space — assuming no prior knowledge.
06
·Deep Learning Basics·★ MEMBER·PAPER·10 min read
A History of Normalization Layers — From BatchNorm to RMSNorm
The layer that made deep learning actually deep, explained from zero. The internal-covariate-shift controversy behind BatchNorm, why LayerNorm threw away the batch axis, and why every modern LLM converged on RMSNorm.
07
·Calculus & Optimization·★ MEMBER·PAPER·9 min read
Convexity and Optimization — Why Deep Learning Works Even Though It Isn't Convex
Optimization textbooks teach a stark divide: convex problems are solvable, non-convex ones come with no guarantees. So why does deep learning — whose loss surface is provably non-convex — work at all? From convex sets and functions to saddle points and flat minima, this article connects the whole story in the language of landscapes.
08
·CNNs & Image Recognition·★ MEMBER·PAPER·8 min read
The CNN Family Tree — From AlexNet to ResNet and EfficientNet
A decade of CNNs told as two campaigns — the race for depth and the race for efficiency. ReLU, residual connections, and compound scaling explained from scratch with metaphors, interactive figures, and code.
09
·Time Series·FREE·11 min read
RNNs and LSTMs from Scratch — Why Learn Them in the Transformer Era
Start from one idea — read a sequence one step at a time while carrying a state — then work out why multiplying the same matrix over and over kills the gradient, and what the three LSTM gates actually fixed. Ends with why Transformers took over, and where this recurrent idea still wins.
10
·Deep Learning Basics·FREE·7 min read
Neural Networks from Scratch — From One Neuron to Many Layers
What a single neuron actually computes, and why stacking layers is pointless without an activation function — shown with a one-line proof that composing linear maps just gives you another linear map. Metaphor, math, an interactive figure, then fifteen lines of numpy.
11
·Deep Learning Basics·★ MEMBER·9 min read
Backpropagation from Scratch — It Is All Just the Chain Rule
Why you can get gradients for ten million parameters for roughly the cost of one forward pass. The chain rule, computational graphs, a two-layer network worked by hand with real numbers, and where vanishing gradients come from — every symbol explained as it appears.
12
·How Transformers Work·FREE·8 min read
Attention from Scratch — The Heart of the Transformer, Explained Visually
Self-attention, the core mechanism behind ChatGPT, explained from zero: analogy, intuition, matrix mechanics, and runnable numpy code. No math background required.