#activation

2 articles

01 ·Deep Learning Basics·FREE·PAPER·13 min read Activation Functions from Scratch — Why Nonlinearity Is Non-Negotiable Without an activation function, a hundred stacked layers can do exactly what one layer does. Starting from that one-line proof, this article traces why sigmoid was abandoned, why ReLU won, and why today's LLMs settled on SiLU and SwiGLU — with an interactive plot where you can drag the input and watch the slope. 02 ·Deep Learning Basics·FREE·7 min read Neural Networks from Scratch — From One Neuron to Many Layers What a single neuron actually computes, and why stacking layers is pointless without an activation function — shown with a one-line proof that composing linear maps just gives you another linear map. Metaphor, math, an interactive figure, then fifteen lines of numpy.