Source Credibility and the Evidence Ladder: How Much Should You Believe a Claim?

Every day we are asked to believe something on the strength of a source: a report, a dashboard, a colleague, a post. Most of the time we decide by instinct. When the decision matters, a little structure helps. This article offers three tools: rate the source and the claim separately, count independent origins rather than repeats, and place the kind of evidence on a ladder. Combined, they turn “I read it somewhere” into a short, honest statement of how much confidence the evidence supports.

Tool 1: separate the source from the claim

We tend to rate a claim by how much we trust whoever said it. But the two are different questions.

• How good is the source? Judged by expertise (could they know?), access (how do they know: first-hand or second-hand?), track record (how often have they been right, and do they correct errors?), and incentives (what do they gain if this is believed?). Research on persuasion since the 1950s has found that people respond to exactly these two qualities: expertise and trustworthiness (Source Credibility).
• How good is the claim? Is it plausible, specific, checkable, consistent with other things we know, and confirmed independently?

The two ratings come apart often. A reliable newspaper passing on a rumour is a good source for “this rumour exists” and a poor one for “the rumour is true”. A source with a poor record sometimes reports a verifiable fact. Intelligence agencies use two separate grading scales, one for the source and one for the information, for this reason.

Also remember that credibility depends on the topic. An excellent hospital researcher may be an ordinary guide to economics.

Tool 2: count independent origins

Five reports of the same claim are impressive only if they have five different origins. Trace each claim back to where it started (Corroboration). Many “confirmations” turn out to be copies or echoes of one starting point (Syndicated Content).

Four reports, two independent origins (agency X, the filing), and one of the “analysts” is a loop. The count that matters is the number of independent routes to the fact, and the strength of each.

The same rule applies to numbers. A figure in a dashboard that is computed from another dashboard is not a second measurement; trace it to its source (Data Lineage).

Tool 3: the evidence ladder

Kinds of evidence differ in how well they guard against error. In medicine, the GRADE approach orders them roughly like this, and the same logic carries over to other fields (Hierarchy of Evidence):

Higher is usually stronger, not always: a well-run observational study can beat a sloppy experiment. The ladder tells you what kind of mistake to look for.

Putting the three together

For a claim you intend to act on, fill in a four-column note:

The columns force three useful habits. You trace the origin instead of counting repeats. You name the kind of evidence instead of saying “the data show”. And you write a confidence you can be held to later (Decision Record, Calibration).

Red flags

• A claim whose source cannot be traced, or is “people familiar with the matter” for every reader.
• A striking figure with no description of how it was counted.
• Many reports that use the same unusual phrase (they probably share an origin).
• A source that benefits from belief and offers no way to check.
• Certainty out of proportion to the evidence (“proves”, “always”, “nobody”).
• A single study treated as settled, or an anecdote treated as a trend.

What to do with low-confidence claims

Low confidence is not a verdict that something is false; it is a verdict that you should not act as though it were true. Options: look for an independent route to confirm it; find out what would settle it cheaply (a look at the filing, a small test); act in a way that can be undone; or wait. The skill is not in believing or rejecting, but in matching the strength of your action to the strength of your evidence.