Precision and Recall

Definition

Precision is the share of returned results that are relevant. Recall is the share of all relevant items that were returned. Precision measures how clean the results are; recall measures how complete they are. Improving one often costs the other.

In plain terms

Ask a search system for everything about a topic. Precision asks: of what you handed me, how much was useful? Recall asks: of everything useful that exists, how much did you find?

With true positives (TP), false positives (FP) and false negatives (FN):

$\text{precision} = \frac{TP}{TP + FP} \qquad \text{recall} = \frac{TP}{TP + FN}$

A system that returns only one perfect result has high precision and poor recall. A system that returns everything has perfect recall and poor precision.

Example

The numbers below are synthetic. A collection holds 100 relevant documents.

Loosening the setting finds more of what exists, at the price of more clutter. Which setting is better depends on what a miss costs compared with what clutter costs.

Common misunderstandings

• “Precision and accuracy are the same.” Accuracy counts every correct decision, including the many easy “not relevant” ones, and can look high when relevant items are rare.
• “One number is enough.” Combined scores such as F1 hide the trade-off. Report both, or the full curve.
• “Recall is easy to measure.” It needs the complete set of relevant items, which is usually unknown and has to be estimated from judged samples.