Regression to the Mean

Definition

Regression to the mean is the tendency of extreme measurements to be followed by less extreme ones, simply because part of any extreme value is chance. It makes the worst performers look improved and the best look declined, even when nothing was done.

In plain terms

Anything measured with some luck in it will, when picked for being extreme, tend to look less extreme next time. The item that topped last month’s chart had good underlying performance and a stroke of luck; the luck does not repeat. Francis Galton noticed this with the heights of parents and children, and it is the reason it is called “regression”.

It is a trap in evaluation. Selecting the worst items, applying a fix and seeing improvement proves nothing without a comparison group that was selected the same way and not treated.

Example

A simulation: 100 items have a true level that varies around 100, and each month’s measured sales add random noise. The 10 best items of month 1 are followed into month 2, with nothing done to them. Averaged over 2,000 repetitions:

The group “lost” 27 points of its lead in a month with no cause at all. Anyone who then credited a competitor, or blamed the team, would be explaining noise.

Common misunderstandings

• “A drop after a peak needs a cause.” Part or all of it is the loss of the lucky component.
• “Applying a fix to the worst items and seeing them improve shows the fix works.” They would have improved anyway; compare with untreated items chosen the same way.
• “It applies only to people or sports.” Any noisy measure, including sales, ratings and ranks, shows it.