Evidlake
Menu

Attrition Bias

Also known as: differential dropout, loss to follow-up.

Bias that arises when units drop out of a study for reasons related to what is measured, so the units that remain no longer represent the original group.

Updated Last reviewed

Synthetic dataExamples on this page use synthetic data created for illustration.

Definition

Attrition bias arises when units leave a study or tracking panel for reasons connected to what is being measured, so the units that remain no longer represent the original group. It is survivorship bias applied to a study that runs over time.

In plain terms

People stop answering surveys, products are withdrawn, sources stop publishing. If leaving is unrelated to the outcome, little is lost. If it is related, for example the worst performers disappear first, the remaining group looks better with every round even though nothing improved.

Remedies start with recording who left and why, and keeping the leavers in the dataset, flagged, so that the comparison can be made with and without them.

Example

100 products are tracked. At the start the average rating is 4.0. By week 8, 20 products have been withdrawn; those 20 averaged 3.2 at the start.

GroupAverage rating at the start
All 100 products4.0
The 20 that left3.2
The 80 that stayed4.2 (since 100 × 4.0 = 400 and 20 × 3.2 = 64, the remaining 336 / 80 = 4.2)

At week 8 the average of the stayers is still 4.2. Compared with the original 4.0 it looks like an improvement of 0.2, but compared with the same 80 products at the start it is no change at all. The “improvement” is entirely attrition.

Common misunderstandings

  • “People and products drop out at random.” Test it by comparing leavers with stayers on what you measured at the start.
  • “Just drop the incomplete units.” That is precisely how the bias gets in.
  • “It only affects surveys.” Any tracked list, such as a panel of products or a watch-list of sources, is exposed.
  • Survivorship Bias

    A selection error from studying only what survived a process and ignoring what dropped out along the way.

  • Panel Data

    Observations of the same units, such as products, firms or people, repeated over time, so change can be studied within each unit.

  • Missing Data

    Values that should exist but were not recorded. Why they are missing decides whether ignoring them is harmless or biases the result.

References

  1. Jerry A. Hausman, David A. Wise. Attrition Bias in Experimental and Panel Data: The Gary Income Maintenance Experiment . Econometrica, 1979. DOI: 10.2307/1914193
  2. Miguel A. Hernán, Sonia Hernández-Díaz, James M. Robins. A Structural Approach to Selection Bias . Epidemiology, 2004. DOI: 10.1097/01.ede.0000135174.63482.43