Subject
Decision Science
Deciding under uncertainty, and keeping a record you can learn from.
Tutorials
- From OODA to Reviewable Decisions: Make Fast Decisions You Can Learn From
John Boyd's observe–orient–decide–act loop rewards speed, but speed only helps if you learn. A short decision record, written at the time, turns each pass through the loop into something you can review fairly.
- Are Your Probabilities Honest? A Practical Guide to Calibration and the Brier Score
Saying 70% is a promise: about seven in ten such calls should come true. How to check that promise with a table, a chart and one number, using twenty forecasts.
- Designing Scores You Can Explain: Rules, Weights, Unknowns and Reasons
A score that cannot say why is a number to be argued about. How to build a scoring scheme with hard rules, weights, a proper place for unknown values, reasons with every result and a check against human judgments.
Glossary terms
- Base Rate
How common something is in the whole relevant population, before looking at any specific evidence.
- Brier Score
The average squared difference between forecast probabilities and what happened. Lower is better; always saying 50% scores 0.25.
- Calibration
Events forecast at p% happen about p% of the time. Calibration is separate from accuracy: an overconfident forecaster is not calibrated.
- Confirmation Bias
Looking for, noticing and remembering evidence that supports what we already believe, and discounting what contradicts it.
- Decision Record
A short, dated note made at the time of a decision: options, evidence, assumptions, the choice and what would change it.
- Explainable Scoring
Returning, with every score, the reasons behind it: which rules fired, what each criterion contributed and what data were missing.
- Framing
Choosing which aspects of a situation to emphasise so as to suggest a particular problem, cause or remedy. The same facts can be framed in different ways.
- Hindsight Bias
Once an outcome is known, believing it was predictable and that you knew it all along. It makes the past look more certain than it was.
- OODA Loop
John Boyd's model of decision-making as a repeating cycle of observing, orienting, deciding and acting. Orientation, how we interpret what we see, is where bias enters.
- Outcome Bias
Judging a decision by how it turned out instead of by how well it was made with what was known at the time.
- Pre-mortem
A planning exercise: imagine the project has already failed and write down why, before starting. Treating failure as certain makes people name risks they would otherwise keep quiet about.
- Proxy Metric
A measurable stand-in for something you cannot measure directly. Proxies work until they become targets: then the number stops tracking what it was meant to represent.
- Weighted Scoring Model
Rates each candidate on several criteria, multiplies each rating by an agreed weight and adds them up. Trade-offs become explicit, and the answer depends on the weights.