Keep this life skill for later
Sign in to save
← All life skills

Learning & thinking · a life skill

Critical Thinking

how sure you are, and how far each fact should move you

Critical thinking isn't doubting everything. It's being able to say how sure you are of a claim, and how much a new piece of evidence should change that. Most of us do the opposite: we hold on to a first impression, or swing all the way over on one striking story. The fix is a habit you can practise: name a number, say in advance what would change it, and then move in small, honest steps.

  • Challenging to start
  • Life-Changing impact
  • Learn it: ongoing
  • One claim at a time
one claim, four pieces of evidence
A confidence ledger for the claim "This supplement improves my sleep.", moving from 60% to 52% in four small steps Written before looking at any evidence: what would change my mind is a week on and a week off with no difference in how I sleep. Starting confidence 60%. A friend swears by it, weighed ×1.2 · a bit likelier if it works: 64%. My tracker: 7 nights on, 7 off, no change, weighed ×0.5 · twice as likely if it doesn't: 47%. A review of trials finds a small effect, weighed ×1.5 · likelier if it works: 57%. Most of those trials paid for by the maker, weighed ×0.8 · a bit likelier if it doesn't: 52%. The biggest move came from the test named in advance, and no single move was more than 17 points. The ratios are made up to show the method. The claim This supplement improves my sleep. Written first · what would change my mind A week on and a week off, with no difference in how I sleep. My gut, before looking anything up 60% 0%50%100% A friend swears by it 64% ×1.2 · a bit likelier if it works My tracker: 7 nights on, 7 off, no change 47% ×0.5 · twice as likely if it doesn't A review of trials finds a small effect 57% ×1.5 · likelier if it works Most of those trials paid for by the maker 52% ×0.8 · a bit likelier if it doesn't From 60% to 52%: still unsure, which is the honest answer here. The ratios are made up to show the method. 1 2 3 4 5

The test written down in advance came in and moved the number most. The friend's story, the kind of evidence that usually moves people furthest, barely moved it at all.

The habit

Try it now.

  1. State a claim that could be wrong

    "This supplement improves my sleep", not "supplements are a scam". If no fact could ever count against it, there's nothing to think about yet.

  2. Put a number on it

    How sure are you, from 0 to 100? A gut number is fine. Writing it down is what lets you notice later whether you moved.

  3. Write what would change your mind, first

    Before you look at anything, name the result that would make you less sure. After you've seen the evidence, it's too easy to decide it doesn't count.

    If you can't name one, you're defending the claim, not testing it.

  4. Ask how much likelier each fact is if it's true

    Not "does this fit?" but "would I see this more if the claim were true than if it were false, and by how much?" A story that fits either way is worth nothing.

  5. Move a little, and keep the record

    Shift your number by what the evidence earns, often a few points. Keep the list, so next time you can see what moved you and whether it should have.

When to break the rule: small steps are for ordinary evidence. When something decisive comes in, the trial you were waiting for, the source admitting it was wrong, move all the way at once. And don't do this for everything: keep it for claims you're about to act on.

Your claim

Weigh one claim, one piece of evidence at a time.

Write a claim you care about, set how sure you are, and say what would change your mind before you add anything. Then add each piece of evidence and judge one thing about it: is it likelier if the claim is true, or if it's false? The chart moves your number by that much, and flags a single move that's suspiciously large.

How sure are you, before any evidence?
50%
The evidence, in the order you met it

    Your confidence

    Write a claim to start

    50%

      Nothing you type is saved or sent anywhere. Copy the record if you want to come back to it when new evidence turns up.

      The study behind small steps

      In a four-year forecasting tournament, the most accurate forecasters changed their minds often, and by small amounts.

      Pavel Atanasov, Philip Tetlock and colleagues looked at how people revised their forecasts over four years of a tournament: more than 400,000 forecasts on almost 500 questions about events whose answers later became known. They looked not at what people believed, but at how they changed it.

      The most accurate forecasters updated often and in small steps. The less skilled tended to do one of two things: confirm their first judgment and barely move, or sit still and then make rare, large jumps. Forecasters who were trained in probabilistic reasoning made smaller updates, and part of the training's benefit ran through that change.

      The honest limits: these were forecasts about world events with a clean answer at the end, and most of the evidence is about which style goes with accuracy, not proof that changing your style alone will make you right. But the style is the part you control.

      A schematic of three ways of updating a forecast as news comes in, not the study's data. Frequent, small updates, the most accurate. Sticks with the first answer, less accurate. Rare, large revisions, less accurate. Frequent, small updates the most accurate what happened Sticks with the first answer less accurate Rare, large revisions less accurate each dot is one forecast, as news comes in
      A schematic of the three styles Atanasov et al. (2020) describe, drawn to show the shape, not their data. Organizational Behavior and Human Decision Processes 160, 19–35.

      The idea

      Why it works.

      A number makes you honest with yourself. "I think so" can absorb any amount of contrary evidence without changing. "60%" can't: after the tracker came back flat, you either moved it or you have to explain why not.

      Asking "how much likelier if true?" is the other half. Most evidence that feels convincing fits the claim, but also fits its opposite nearly as well. A friend's good night would happen with or without the pill. Asking the question shrinks that evidence to what it's worth, and lets the rare decisive test stand out.

      When it goes wrong

      Habits that help.

      • Watch for the striking story. One vivid case moves people further than a dull table of a hundred. Ask how often you'd hear that story if the claim were false.
      • Count sources, not repeats. Five articles quoting the same study are one piece of evidence. Weigh it once.
      • Notice when you won't move. If three pieces of evidence against a claim leave your number where it was, the problem isn't the evidence.
      • Ask what the other side would point to. Before you settle, find the best piece of evidence against your view and weigh it as fairly as the ones for it.

      Go deeper

      Keep going.

      Keep this going in the app.

      HourLife turns these ideas into a daily system — goals, habits, focus, and reflection in one private place.

      Download on the App Store Download on the Mac App Store See everything HourLife does