Medical Imaging AI — Validation Design Comes Before the Accuracy Number

An AI that scored beautifully on your own hospital's data turns out to be useless at the hospital next door — a failure that has been reported over and over in medical imaging. Sensitivity and specificity, the way prevalence flips the meaning of a positive result, internal versus external validation, what site-to-site variation actually is, and what regulators are really reviewing: building up, from first principles, the habit of questioning the validation design before the accuracy figure.

"It works here, but not at the hospital down the road"

Say you have built an AI that flags suspected pneumonia on chest X-rays. Measured on your own hospital's data, the accuracy is everything you could ask for. Then you take the same model to another hospital and the performance visibly falls apart. In medical imaging AI this is not a rare anecdote — it is a pattern that has been reported again and again.

A validation study published in 2018 showed that a pneumonia detection model trained on chest X-rays from one group of hospitals scored well on data from the hospitals it had trained on, but did worse on data from a different hospital — and that the same model could tell, from the image alone and with high accuracy, which hospital a scan had been taken at. The model had learned not only the shadow of pneumonia but each hospital's characteristic way of rendering an image.

Knowing how convolutional networks work is not enough to prevent this failure. Taking medical imaging as the case study, this article builds up from scratch the habit of questioning the validation design before the accuracy number, in this order: an analogy, then sensitivity and specificity, then prevalence, then external validation, then regulation.

An analogy: the student who memorized the past papers

Picture a student who gets full marks on every test their school sets. They look like a strong student — but they may simply have memorized that school's past papers. No amount of re-testing with questions from the same school can tell the two apart. Only a mock exam written by someone else — questions from a different school — reveals whether they understood the material or merely memorized it.

Validating a medical imaging AI has exactly the same shape. Splitting your own hospital's data into training and test portions and measuring on the held-out part is a re-test with the same school's questions; only measuring on data from another institution is the mock exam. The first is called internal validation, the second external validation. However high the internal number, it alone cannot separate "the model is looking at the disease" from "the model has memorized how images look here."

Intuition: a model can learn the "look" instead of the disease

Why does memorization happen? Because a neural network is not fussy about the means as long as the loss goes down. If the training data contains a shortcut that happens to correlate with the presence of disease, the model will seize on that before it engages with the lesion itself. This is known as shortcut learning, and medical images offer plenty of raw material for it.

• Equipment and acquisition settings: X-ray and CT vendors, dose, and reconstruction settings differ from site to site, giving each institution's images their own texture (noise grain, contrast).
• Circumstances of the scan: severely ill patients tend to be imaged with a bedside portable unit. So "the look particular to portable images" correlates with "severely ill," and the model can use traces of the equipment as a sign of disease.
• Patient population: the people who come through a university hospital differ completely in age and severity from those who visit a screening center.
• Labelling habits: how a diagnosis is worded and how a radiology report is written depend on institutional culture.

This gap between the training data and the deployment site is collectively called domain shift — in this context, site-to-site variation. The divergence between internal and external validation is the same structure as the divergence between training error and test error in overfitting, except that it happens across the boundary of "institution" rather than "data split."