In 1943, McCulloch and Pitts modeled the nerve cell with a piece of simple arithmetic. Weight each input, add the results, and fire if the total crosses some threshold. That "multiply and add" is the basic operation that has not changed in the 80 years since.

In 1958, Rosenblatt made it learn, publishing the perceptron. Nudge the weights a little whenever the answer is wrong, repeat, and the thing learns to classify. Expectations ran high at the time.

But in 1969, Minsky and Papert pointed out the limits. A single-layer perceptron cannot learn even XOR (exclusive or), because it cannot solve a problem that no single straight line can split. This brought on the first winter.

That way is backpropagation, popularized by Rumelhart and colleagues in 1986. It takes the error at the output and distributes it backward toward the inputs using the chain rule of differentiation. This made training multilayer networks possible.

Yet progress stalled again. Make the layers deep and the gradient shrinks as the error travels backward until it disappears altogether (vanishing gradients). On top of that, neither the compute nor the data was there. Through the 1990s and 2000s, neural networks fell out of the mainstream both in academia and in practice. The handful of people who kept plodding away at the research during those years were the ones who set up the explosion to come.