What Machine Learning Really Is — Understanding “Learning” Without the Math

What actually separates writing a program from training a model. Rules versus examples, memorization versus generalization, and a map of supervised, unsupervised, and reinforcement learning — with almost no math.

From writing rules to showing examples

Say you have to build a spam filter. The obvious approach is to write down conditions.

"If the subject line contains You've won, it's spam." "If it comes from an unknown address and has three or more links, it's spam." Building software by hand-writing conditions like these is called a rule-based approach. The first few dozen cases work beautifully. Then senders adapt. You've won becomes You have won, then Y0u won, then it hides inside an image. Your rule file swells to thousands of lines, and nobody remembers which rule was added for what.

Machine learning turns the problem inside out. Instead of writing the rules, you show the machine a large pile of labelled examples and let it construct the rule itself. Hand it ten thousand spam messages and ten thousand normal ones, and say: find the boundary between these two piles. Your job shifts from author of rules to curator of examples and judge of results.

The difference is clearest if you look at what goes in and what comes out.

Deriving the rules from the answers — that is what "learning" means here.

The intuition: it's a game of guessing a function

Let's make that concrete. Assume there exists some true function that takes an email and returns spam or normal. Nobody knows what is inside it. All you have is a pile of input–output pairs it has produced.

The machine's job is straightforward. It starts with a function of a fixed shape that has a great many knobs (parameters) on it. At first the knobs are set randomly, so the predictions are junk. You show it an example, and when the prediction misses the answer, you turn the knobs slightly in the direction that shrinks the miss. Repeat that tens of thousands of times and the function creeps toward the true one.

In symbols: you are searching for an $f$ such that $y \approx f(x)$, where $x$ is the input and $y$ is the answer. That is the only equation in this article. Learning is nothing more than turning knobs until the function fits the examples — if you take away one sentence, take that one.

Which way and how far to turn each knob is decided by a loss function and gradient descent, the subject of the next article (Loss Functions and Optimization).

Memorizing is not generalizing

Here comes the single most important idea in the field: generalization.

Picture a student who has memorized every past exam paper. Perfect score on the past papers, helpless the moment a number changes. Models do exactly the same thing. A model that reproduces the training examples flawlessly but misses on data it has never seen is worth nothing in production.

So the target is never fidelity to the examples you showed; it is accuracy on data the model has not seen. One iron rule follows:

Always evaluate on data that was not used for training.

Accuracy measured on the training set is the score a student gets on the paper they memorized — not evidence of ability. Overfitting and Evaluation Design takes this trap apart in detail.

A map of the three kinds of learning

Machine learning splits into three families, depending on what kind of data you can supply.

Supervised learning is the case where every input comes paired with its correct answer: "this image is a cat", "this apartment sold for 320,000". A human teacher supplied the labels. When the answer is a category it is classification; when it is a continuous number it is regression. This is by far the most common setting in industry.

Unsupervised learning has no labels at all — it looks for structure in the data itself. Clustering groups similar customers together automatically; dimensionality reduction squeezes high-dimensional data into two dimensions so you can look at it. Since nobody defined what "correct" means, judging whether the result is useful stays a human job.

A related case deserves its own name. When a large language model is pretrained to predict the next word, there is a correct answer for every position — but no human wrote it down; it was generated from the text itself. This is self-supervised learning, and the fact that it needs no labelling budget is precisely what made training on internet-scale text possible.

Reinforcement learning replaces answers with rewards. In a board game, nobody knows the correct move at step 37, but at the end you get +1 for a win and −1 for a loss. The system tries things, observes rewards, and learns a policy that maximizes the long-run total. Robotics, game playing, and RLHF — aligning an LLM with human preferences — all live here.

Comparing things by direction: the idea of similarity

Whatever the input — text, image, audio — a model eventually turns it into a list of numbers, a vector. Once everything is a vector, "similar" becomes something you can compute. The most basic tool for that is the dot product.

Picture two arrows. The more their directions agree, the larger the dot product; at right angles it is zero; pointing opposite ways it goes negative. Strip out the effect of arrow length, keeping only the agreement in direction, and you have cosine similarity. Spin the two vectors below and watch the numbers move.

"Similar things sit near each other." That deceptively simple idea underpins search, recommendation, embeddings, and — as you will see much later — the attention mechanism inside Transformers.

The smallest possible machine learning, in code

Here is a learner with no knobs at all: nearest neighbour. When a new item arrives, find the most similar item you already have and reuse its answer.

import numpy as np

def cosine(a, b):
    return a @ b / (np.linalg.norm(a) * np.linalg.norm(b))

def predict(x, X_train, y_train):
    sims = [cosine(x, xi) for xi in X_train]   # similarity to every training point
    return y_train[int(np.argmax(sims))]       # reuse the answer of the closest one

That is a legitimate program that produces answers from data. But it memorizes the training set wholesale: it slows down as data grows, and a little noise makes its answers jump around. Moving from "keep every example" to "summarize the examples into a set of parameters" is, in a sentence, the history of machine learning.

Three things that matter in practice

1. First ask whether rules would do If ten conditions solve 80% of your cases, you may not need machine learning at all. A trained model carries ongoing costs — data collection, labelling, evaluation, retraining — while rules are explainable and can be fixed the same afternoon.

2. How you label the data is the problem definition If two annotators disagree about what counts as spam, that disagreement becomes the ceiling on your model's accuracy. Data is a more honest specification than any document. When accuracy stalls, suspect the labels before the architecture.

3. An accuracy number means nothing on its own Whether "95% accurate" is good depends entirely on the baseline. If 95% of email is normal, then a program that answers "normal" every single time also scores 95%. The third article in this series dissects that trap.

Summary

• Machine learning replaces "write the rules" with "derive the rules from labelled examples"
• Learning itself is just turning knobs (parameters) until a function fits the data
• The goal is never to reproduce the training set — it is to generalize to unseen data
• Supervised, unsupervised/self-supervised, and reinforcement learning differ in what data you can give

Next up: how the machine decides which way and how far to turn each knob — loss functions and gradient descent, explained with equations, code, and a figure you can drag.