Evidlake
Menu

Outcome Bias

Also known as: resulting.

Judging a decision by how it turned out instead of by how well it was made with what was known at the time.

Updated Last reviewed

Synthetic dataExamples on this page use synthetic data created for illustration.

Not adviceThis is educational material. It is not investment, business or other professional advice.

Definition

Outcome bias is judging the quality of a decision by how it turned out rather than by how well it was made with the information available at the time. Good decisions can have bad outcomes and bad decisions can be lucky, so a single outcome is a noisy teacher.

In plain terms

In experiments by Jonathan Baron and John Hershey, people rated the same decision, made with the same information, as better when it ended well and worse when it ended badly. Poker players call this “resulting”.

The way out is to judge the process: what was known, which options were considered, how likely success was thought to be, and whether the reasoning was sound. That is only possible if the reasoning was written down at the time (see Decision Record), since memory rewrites it once the outcome is known (Hindsight Bias).

Example

DecisionEstimated chance of success when decidedOutcomeNaive verdictProcess verdict
A70%, evidence soundFailedBad decisionGood decision, unlucky
B20%, little evidenceSucceededGood decisionPoor decision, lucky

Repeated 100 times, decisions like A succeed about 70 times and decisions like B about 20. Judging each by its single outcome would teach the team to make more decisions like B.

Common misunderstandings

  • “A good result proves a good decision.” It may prove only that the odds were kind.
  • “Outcomes do not matter.” Over many decisions they matter greatly; the point is not to over-read any one of them.
  • “It is a failing of individuals.” Organisations that reward and blame by outcome alone build it into their culture.
  • Hindsight Bias

    Once an outcome is known, believing it was predictable and that you knew it all along. It makes the past look more certain than it was.

  • Decision Record

    A short, dated note made at the time of a decision: options, evidence, assumptions, the choice and what would change it.

  • Calibration

    Events forecast at p% happen about p% of the time. Calibration is separate from accuracy: an overconfident forecaster is not calibrated.

References

  1. Jonathan Baron, John C. Hershey. Outcome Bias in Decision Evaluation . Journal of Personality and Social Psychology, 1988. DOI: 10.1037/0022-3514.54.4.569
  2. Annie Duke. Thinking in Bets: Making Smarter Decisions When You Don't Have All the Facts . Portfolio, 2018. ISBN 978-0735216358.