Survivorship Bias

Definition

Survivorship bias is a selection error that arises when an analysis looks only at things that survived some process and ignores those that dropped out. Because the failures are missing, the surviving group looks healthier, safer or better than the full original group really was.

In plain terms

The dropouts do not announce themselves. Companies that went bankrupt leave no annual reports on the shelf, projects that failed are not written up, and the products that were discontinued are no longer for sale. What remains is a flattering sample.

A well-known account comes from the Second World War. Analysts examined returning aircraft and mapped where they had been hit. Abraham Wald pointed out that the map showed where a plane could be hit and still return. The areas with no holes were the dangerous ones, because planes hit there did not come back.

Example

The numbers below are synthetic.

A study that only reads today’s list of open funds reports 6.0%. The true average for everything that started is 4.4%, so the survivors-only figure overstates the result by 1.6 percentage points.

Common misunderstandings

• “More data fixes it.” A larger sample of survivors is still a sample of survivors.
• “It is a finance problem.” It appears wherever failures are removed from view: medicine, startups, product reviews, career advice.
• “I can adjust for it afterwards.” Only if you can recover the dropouts. If they were deleted, the honest answer is to say the result cannot be trusted.