Deep Learning Basics
Neural nets, backprop, optimization, regularization
01
·Deep Learning Basics·FREE·7 min read
Neural Networks from Scratch — From One Neuron to Many Layers
What a single neuron actually computes, and why stacking layers is pointless without an activation function — shown with a one-line proof that composing linear maps just gives you another linear map. Metaphor, math, an interactive figure, then fifteen lines of numpy.
02
·Deep Learning Basics·★ MEMBER·9 min read
Backpropagation from Scratch — It Is All Just the Chain Rule
Why you can get gradients for ten million parameters for roughly the cost of one forward pass. The chain rule, computational graphs, a two-layer network worked by hand with real numbers, and where vanishing gradients come from — every symbol explained as it appears.
03
·Deep Learning Basics·★ MEMBER·PAPER·10 min read
A History of Normalization Layers — From BatchNorm to RMSNorm
The layer that made deep learning actually deep, explained from zero. The internal-covariate-shift controversy behind BatchNorm, why LayerNorm threw away the batch axis, and why every modern LLM converged on RMSNorm.
04
·Deep Learning Basics·FREE·PAPER·13 min read
Activation Functions from Scratch — Why Nonlinearity Is Non-Negotiable
Without an activation function, a hundred stacked layers can do exactly what one layer does. Starting from that one-line proof, this article traces why sigmoid was abandoned, why ReLU won, and why today's LLMs settled on SiLU and SwiGLU — with an interactive plot where you can drag the input and watch the slope.
05
·Deep Learning Basics·★ MEMBER·PAPER·15 min read
Weight Initialization and Regularization — What Lets Training Start, and What Keeps It Going
The same architecture will train or refuse to train depending on nothing more than how large the random numbers in its weights were. Starting from how variance propagates, this piece derives Xavier and He initialization, then turns to weight decay and dropout as the conditions that keep training going — with interactive figures, PyTorch code and the mistakes that actually cost people days.
06
·Deep Learning Basics·★ MEMBER·PAPER·12 min read
Hyperparameter Search — Hunches, Grids, and Bayesian Optimization
Gradients tell you nothing about the learning rate, so you have to go looking. Why grid search is weak, why search spaces should be carved on a log scale, what a Bayesian acquisition function is actually counting, and why early stopping beats a cleverer search algorithm — with Optuna code and the traps that bite in production.
07
·Deep Learning Basics·★ MEMBER·PAPER·12 min read
Graph Neural Networks from Scratch — Learning from Connections
Social networks, molecules and road maps are all built from dots and lines. This piece builds message passing up from nothing, explains why stacking layers makes every node look identical (over-smoothing), and follows the idea through to how recommenders and drug discovery actually use it.