Symbolic vs. Connectionist — Where a 60-Year Argument Stands Today

AI has always had two tribes: one that writes the rules down, one that learns weights from examples. Tracing the Perceptrons book, the collapse of expert systems, and the backpropagation comeback explains why today's LLMs still call calculators and get shackled to grammars.

The recipe writer and the taste tester

There are roughly two ways to teach someone to cook.

One is to write the recipe down: three grams of salt, twenty minutes at 180°C, pull it out when the surface browns. Every step is explicit, so anyone following it gets the same dish — and when the dish fails you can point at the line that was wrong.

The other is to stand next to them while they cook it fifty times. You don't explain anything. They cook, you taste, you say "more salt" or "heat's too high." Eventually their hands know what to do, and neither of you can articulate why.

Those two teaching styles map exactly onto AI's two schools: symbolic AI and connectionism. Sixty years of the field is largely the story of these tribes telling each other that the other approach could never reach intelligence.

Today it looks like a rout in favor of the second. Except that the winner — the large language model — calls out to a calculator whenever it has to multiply, gets its output shackled to a JSON schema, and is paired with a symbolic deduction engine when it does mathematics. The side that won has not put down the loser's tools. This article is about why.

Symbolic AI — thinking is rearranging symbols

When the phrase "artificial intelligence" was coined at the 1956 Dartmouth workshop, the symbolic camp owned the field. Allen Newell and Herbert Simon's Logic Theorist proved theorems from Russell and Whitehead's Principia Mathematica on its own — doing what a human mathematician does, but recast as a search over symbol-rewriting rules.

In their 1976 Turing Award lecture they compressed the position into a single claim: a physical symbol system has the necessary and sufficient means for general intelligent action. In plain terms: represent the world as symbols, rearrange them by rules, search for promising arrangements. That's enough for intelligence — and anything intelligent must be doing that underneath.

The strengths of this position have not aged. If the rules are right, the answer is provably right. You can show why, as a chain of rule applications. It needs almost no data. And it composes without limit: a system that understands "Alice hit Bob" automatically handles "Bob hit Alice," because all it did was swap two symbols.

Connectionism — thinking is the strength of connections

The other camp starts from wiring rather than rules. In 1943 McCulloch and Pitts formalized a neuron as an element that fires when the weighted sum of its inputs crosses a threshold. In 1958 Frank Rosenblatt made that element learn and called it the perceptron. It was built in hardware as the Mark I Perceptron, and the press of the day described it as the embryo of a machine that would walk, talk, see and write.

The whole thing is startlingly simple.

In words, multiply each input by its weight, add them all up, add the offset $b$, and output 1 if the total is zero or above and 0 otherwise.

Equation (1) says: multiply each input $x_i$ by its weight $w_i$, add them all up, add an offset $b$, and output 1 if the result is positive and 0 if it isn't. The $x_i$ are the pieces of the input (pixel brightnesses, say), the $w_i$ say how much each piece matters, and $b$ is the dial for how easily the whole thing fires at all.

The learning rule is one line too.

$$w_i \leftarrow w_i + \eta\,(t - y)\,x_i$$

In words, when the output $y$ already matches the correct answer $t$, nothing changes; when it is wrong, each weight is nudged a little, by a step of size $\eta$, in the direction of the mistake.

Here $t$ is the correct answer, $y$ is what came out, and $\eta$ ("eta") is the step size — how far to move per correction. When the output is already right, $t - y = 0$ and nothing happens; only mistakes cause a nudge, in the direction of the mistake. That is enough: if the data can be separated by a line at all, this rule provably finds a separating line in finitely many steps.

This is the decisive break with the symbolic camp. Nobody wrote any rules. Examples went in, and a pile of numbers took over the job that rules used to do.

Then in 1969 a book came out

Marvin Minsky and Seymour Papert's Perceptrons: An Introduction to Computational Geometry (MIT Press, 1969; expanded edition 1988) answered the hype with mathematics.

Their weapon was exclusive OR (XOR) on two inputs: output 1 when exactly one input is 1, and 0 when both are 0 or both are 1.

That a single-layer perceptron can never represent this takes three lines. Substituting the four rows into equation (1) requires $b < 0$, $w_1 + b \ge 0$, $w_2 + b \ge 0$, and $w_1 + w_2 + b < 0$ to hold at once. The middle two give $w_1 \ge -b$ and $w_2 \ge -b$, so $w_1 + w_2 + b \ge -b$. But $b < 0$ makes $-b$ positive, so $w_1 + w_2 + b$ is positive — contradicting the fourth condition.

Geometrically it's obvious. A single-layer perceptron can only draw one straight line and split the plane in two. XOR's four points have matching colors on the diagonals, so no line works.

The real substance of the book, though, lies past XOR. Minsky and Papert formalized how much of the input a perceptron can look at in one go as the order of a predicate, then proved that computing parity (is the number of 1s even or odd?) requires a unit that sees the entire input at once, and that deciding whether a shape is connected requires unbounded order. Summing local features cannot, in principle, capture global properties. That argument still echoes in debates about convolutional networks today.

The obvious reply is: use more layers. Minsky and Papert addressed that directly. They granted that multilayer networks are more powerful in principle, and pointed out that no learning rule for them existed. They wrote that testing — or rejecting — their intuition that the multilayer extension would prove sterile was an important open research problem.

You often hear that this one book killed neural network research and caused the AI winter. That's too tidy; funding shifts had many causes and no single book carries them. What is clear is that the technical point was correct. There was no learning rule for multilayer networks. It took seventeen years to produce one.

The symbolic golden age, and its three walls

Those seventeen years were symbolic AI's peak. DENDRAL, started in 1965, inferred molecular structures from mass spectrometry data. MYCIN, built at Stanford in the 1970s, diagnosed bacterial infections and recommended antibiotics, and evaluations rated its recommendations as comparable to specialists' (it was never deployed clinically). XCON (R1), which DEC put into production around 1980, configured VAX orders from rules and became one of the first expert systems to demonstrably save money. Japan's Fifth Generation Computer Systems project (1982–1992) was a national bet on the same logic-programming foundation.

Edward Feigenbaum's slogan was that the power is in the knowledge — not in clever search, but in thousands of IF-THEN rules extracted from experts. He was right, up to three walls.

Wall 1: the knowledge acquisition bottleneck. Experts cannot explain their own judgments. A veteran inspector spots a defective part instantly and cannot write down the rule; the rule they do write down usually disagrees with what they actually do. Converting knowledge into rules became the single largest cost of building these systems.

Wall 2: combinatorial explosion. Add rules and the interactions between them grow with the square of the count, while the states to search grow exponentially with depth. The nasty thing about exponentials is that "wait for a faster machine" is not an answer.

Wall 3: brittleness. Step one inch outside the rules' intended scope and an expert system doesn't say "I don't know" — it returns a confident, meaningless answer. A human would notice being out of their depth; a chain of rules just keeps firing.

Douglas Lenat's Cyc, begun in 1984, attacked brittleness head-on: write down all of common sense by hand. Millions of assertions were entered manually, and it still didn't cover the ground. When the LISP machine market collapsed at the end of the 1980s, the second AI winter arrived.

The connectionist comeback — backpropagation and scale

In 1986 Rumelhart, Hinton and Williams published backpropagation in Nature, answering the seventeen-year-old homework problem. By propagating output error backward layer by layer, the weights of a multilayer network can be updated by gradient descent. What the paper emphasized was not the arithmetic but the consequence: hidden layers invent useful internal representations on their own, with nobody specifying them. (Historically, Linnainmaa's 1970 reverse-mode automatic differentiation and Werbos's 1974 thesis got there first.)

XOR falls to two hidden units. The mechanics are worked through in Building a neural network from scratch.

Even so, the 1990s belonged not to neural networks but to support vector machines and kernel methods — simply because there wasn't enough data or compute. The tide turned with ImageNet in 2012, after which "make it bigger and feed it more" became the dominant strategy. Rich Sutton's 2019 essay The Bitter Lesson states the pattern bluntly: methods that hand-encode human knowledge win in the short run and always lose in the long run to general methods that exploit computation.

So is it settled?

No — because each camp still owes the other an answer.

The bill against connectionism was presented in 1988 by Fodor and Pylyshyn as the systematicity argument. Anyone who understands "Alice loves Bob" understands "Bob loves Alice" with no further learning. A symbolic system gets that for free from structure; a distributed representation has no guarantee of it. Harnad's symbol grounding problem (1990) came from a nearby angle: defining symbols in terms of other symbols is looking words up in a dictionary written in the same dictionary, and meaning never enters.

Modern evidence flows into the same channel. Multiplication with more digits, multi-step compositional reasoning, and word problems where only the numbers or names were swapped all produce oddly sharp drops in large-model accuracy. Inside the training distribution the interpolation is beautiful; a short step outside it, things break. Why the breakage looks the way it does is covered in The mechanics of hallucination.

Symbolic AI has an unpaid bill too, and it's the bigger one: nobody knows how to carve the world into symbols in the first place. You can write rules that manipulate the symbol "cat," but deciding that this particular array of pixels is a cat has always fallen back on learning. That is Moravec's paradox — logic is easy, perception and movement are hard.

So here is where we actually stand: perception and association went decisively to the connectionists; verification and guarantees are still symbolic territory.

Neurosymbolic — the third road

Henry Kautz's 2020 AAAI lecture laid out a taxonomy of ways to combine the two. In practice, though, almost everything you meet in production is one of three shapes.

1. The neural side calls the symbolic side (tool use). Calculators, code interpreters, SQL, SAT solvers, unit conversion. The model decides what should be computed; the computing is outsourced to something exact. LLM function calling is this pattern, mass produced.

2. The symbolic side calls the neural side (search heuristics). AlphaGo dropped neural networks — proposing candidate moves and estimating position quality — into the middle of a classical tree search. DeepMind's AlphaGeometry has the same skeleton: a language model proposes auxiliary constructions and a symbolic deduction engine carries the proof. Ideas from the neural half, correctness from the symbolic half.

3. The symbolic side constrains the neural side (constrained generation). A grammar or JSON schema restricts which tokens may be emitted at each step. The form of the output is guaranteed; the content stays probabilistic. This is the most widely deployed and least discussed neurosymbolic technique in the industry.

Research on differentiable logic programming — DeepProbLog and relatives, where the logic itself is learned by gradient descent — is lively, but the three patterns above dominate by volume. Worth adding: retrieval-augmented generation is part of the same lineage, since it consults an external symbolic store before answering (Graph RAG leans harder on the symbolic structure).

How this gets used on the job

Who touches it, and when. Anyone building an LLM application that feeds model output into a downstream system draws this boundary on every feature. Deciding how much to leave to the model and where to hand off to something exact is the design work.

Parameters and tools you'll actually name. For output-format guarantees: structured outputs, or grammar-constrained decoding — llama.cpp's GBNF grammars, Outlines, guidance, vLLM's guided decoding. For arithmetic, dates and unit conversion: function calling / tool use. For business rules: a rules engine, a decision table, or plain SQL. When a knowledge graph is in the mix, put entity normalization and linking on the symbolic side and natural-language interpretation on the neural side.

Failure modes that bite.

1. "I asked for JSON in the prompt" is not a guarantee. It's a request, and longer outputs break it more often. At minimum, validate with a parser and retry; where you can, mask tokens with a real grammar.
2. Adding tools moves the failure mode from "wrong answer" to "right tool, wrong arguments." Ship argument schema validation together with tool errors returned in a form the model can read, or wrong results will flow downstream quietly.
3. Never keep rules in two places. Pricing or credit conditions written into both a decision table and a prompt will drift when only one gets updated. Keep one source of truth, and keep it out of the prompt.
4. "Adding symbols makes it explainable" is only half true. What you can explain is what happened after the tool call; why the model chose that tool with those arguments stays inside the model. If you have audit requirements, the decision itself has to live on the rules side.

How this shows up in interviews and design reviews. "Should an LLM do arithmetic?" — no, because multi-step carrying degrades outside the training distribution and degrades confidently; route it to a tool. "Why can't a single layer compute XOR?" — it isn't linearly separable, and the three-line proof above is the answer.

Takeaways

• Symbolic AI writes the rules; connectionism learns weights from examples. The whole sixty-year argument sits on that one line
• The 1969 critique was correct — but it was specifically about the absence of a multilayer learning rule, and that was solved in 1986
• Even after connectionism swept perception and language, verification, guarantees and compositional exactness stayed on the symbolic side
• Essentially every production system is a hybrid. The question moved from "which side wins" to "where do we draw the line"

You don't have to pick a tribe. For each feature in front of you, decide whether it needs certainty or flexibility. That's the most useful thing sixty years of argument has to offer.