Confidence Interval

Definition

A confidence interval is a range calculated from a sample so that, if the same procedure were repeated on many samples, a stated share of the ranges (often 95%) would contain the true value. It expresses the precision of an estimate, not the probability that this one range is right.

In plain terms

An estimate without a statement of precision is half a number. “The average rating is 4.1” could mean anywhere from 3.9 to 4.3 or from 3.0 to 5.0. The interval says which. For a mean, the usual recipe is the sample mean plus or minus about two standard errors, where the standard error is the standard deviation divided by the square root of the sample size.

Precision grows slowly with data: four times as many observations halve the width. The interval says nothing about bias: a precise estimate from a badly chosen sample (Sampling Frame) is precisely wrong.

Example

Ratings with a standard deviation of 0.8 and a sample mean of 4.1.

Four times the data halves the interval. The interval describes the average rating, not the range in which individual ratings fall (most of those lie between about 2.5 and 5).

Common misunderstandings

• “There is a 95% chance the true value lies in this particular interval.” The 95% belongs to the method: across repetitions, 95% of such intervals capture the truth.
• “A narrow interval means an accurate result.” It means a precise one; bias can still shift it away from the truth.
• “The interval shows where individual values lie.” It concerns the average; the spread of individuals is a different range.