Proxy Metric
Also known as: proxy measure, Goodhart's law, surrogate metric.
A measurable stand-in for something you cannot measure directly. Proxies work until they become targets: then the number stops tracking what it was meant to represent.
Synthetic dataExamples on this page use synthetic data created for illustration.
Not adviceThis is educational material. It is not investment, business or other professional advice.
Definition
A proxy metric is a measurable stand-in for something you care about but cannot measure directly, such as review counts for sales. Proxies work until they become targets: when people optimise the number itself, it stops tracking what it was meant to represent.
In plain terms
Sales cannot be seen, so we look at a sales rank; satisfaction cannot be seen, so we look at star ratings; interest cannot be seen, so we count clicks. A proxy is only as good as its link to the real thing, and that link was established under particular conditions.
Goodhart’s law, in a common phrasing, says that when a measure becomes a target it ceases to be a good measure. Part of the reason is gaming; part is that picking the items with the best proxy value selects luck and noise as well as the real thing (see Regression to the Mean). Defences: know what the proxy is a proxy for, test the link again when incentives change, use several proxies, and measure the target directly when the decision is important.
Example
Review count as a proxy for sales, tracked across a year (synthetic):
| Period | What changed | Correlation between review count and sales |
|---|---|---|
| Before | Reviews arrive naturally | 0.80 |
| After | A seller programme starts paying for reviews | 0.40 |
The proxy did not become less visible; it became less informative, because the number of reviews now partly reflects who is buying them. A screen built on “many reviews means strong demand” quietly lost most of its meaning.
Common misunderstandings
- “A highly correlated proxy is as good as the real thing for ever.” The link holds under the conditions it was measured in.
- “Goodhart’s law is only about cheating.” Honest optimisation and selection effects also break the link.
- “In a report, proxy and target can be named interchangeably.” Call a rank a rank and not “sales”.
References
- David Manheim, Scott Garrabrant. Categorizing Variants of Goodhart's Law . arXiv, 2018.
- Jerry Z. Muller. The Tyranny of Metrics . Princeton University Press, 2018. ISBN 978-0691174952.