Keep this life skill for later
Sign in to save
← All life skills

Learning & thinking · a life skill

Advanced Bias Spotting

an audit for the decisions you can't take back

Catching a bias in the moment is a habit. This is the system around it, for decisions with real stakes: a short audit that goes looking for what you'd see if you were wrong, starting with the cases that went fine, which are the ones nobody thinks to check. Knowing when to skip it matters as much. Most decisions are small and easy to undo, and auditing them spends the attention the big ones need.

  • Challenging to start
  • Life-Changing impact
  • Learn it in 30 days
  • Audit only what you can't undo
the batches nobody looked at
A bakery's dough, plotted by the temperature of the room it proved in: the flat batches alone show no pattern, and every batch shows one The decision: keep proving dough in the cold back room this winter? Big, and hard to undo. If the room is to blame, there should be more flat loaves on cold days than warm ones. Left panel, only the 8 flat batches, spread from 14.5 to 25 degrees: no pattern. Right panel, every batch: the 12 good ones were all at 19.5 degrees or warmer. Below 19 degrees, 3 of 3 batches came out flat. At 19 and up, 5 of 17. Decision: move the dough to the kitchen until spring. The decision Keep proving dough in the cold back room this winter? Big · hard to undo If the room's to blame, I'd see more flat loaves on cold days than on warm ones. Only the flat batches 14° 18° 22° 26° room temperature, °C Every batch 14° 18° 22° 26° room temperature, °C Flat on cold days and warm ones. The good ones were all warm. Below 19° 3 of 3 flat 19° and up 5 of 17 flat Decided, and why Move the dough to the kitchen until spring. Every cold batch failed. Undo it in March if I'm wrong. 1 2 3 4 5

The flat batches alone look like bad luck. Put the good batches beside them and every batch below 19° failed. A made-up bakery; the study below used a business case built the same way.

The system

Try it now.

  1. Sort by stakes first

    Is it big if you're wrong, and hard to undo? Both: run the full audit. One of the two: run the first two checks. Neither: decide and move on.

  2. Write what you'd see if you're wrong

    Not the evidence for your view: the pattern that would have to show up if your view were true, written so it could fail.

  3. Pull up the cases that went fine

    The failures come to you; the successes don't. Find the hires who worked out, the launches that landed, the batches that rose.

    This is the check the study below trained.

  4. Compare rates, not counts

    Five failures on warm days sounds like more than three on cold ones, until you know how many of each there were.

  5. Decide, and write down why

    One line: the call, the reason, and what would make you undo it. That line is what you check against later.

When to break the rule: a small decision you can undo tomorrow. Make it and let the outcome teach you. And an audit that turns up one doubt hasn't overturned anything; a view can survive its audit. Last, keep it for your own decisions. Telling someone else they're biased is the blind spot, pointed outward.

Decision audit

Size the audit, then run it.

Name the decision and answer two questions about it. The audit grows with the stakes: nothing for small choices you can undo, two checks for the middle, and six for decisions that are big and hard to take back.

If it's wrong, the cost is
Undoing it is

Your audit

Nothing you type is saved or sent anywhere. Copy the audit into the notes you keep with the decision, so you can check it when the outcome comes in.

Does training carry over to a real-world decision?

Weeks after one training game, fewer students chose the answer that only the confirming evidence supported.

Anne-Laure Sellier, Irene Scopelliti and Carey Morewedge followed 290 graduate students in three professional programs. Each played one session of a bias-training game, 80 to 100 minutes long. Because of how the school timetabled it, some played before an unannounced business case in class and some after, so the case caught some of them trained and some not.

The case was modelled on the decision to launch the Challenger shuttle: whether to race a car whose engine had failed several times. Looking only at the races with failures, temperature seemed not to matter. To see the problem, you had to set them beside the races with no failure at all, which is what the bakery above does.

Fewer trained students chose to race. The effect was no weaker when the case came 43 to 52 days after training than when it came within 11. The trained students were also less confident, but that wasn't what made the difference: they did less confirmatory hypothesis testing, and that explained the better choices.

The honest limits: it's one field experiment and one case. The groups came from the school's schedule rather than being randomly assigned, which the authors addressed with controls. And most trained students still chose to race, so the training helped without curing anything, which is why the audit asks you to do the looking yourself.

Chose to race

The choice you reach by looking only at the failures

Case before the training
72.2%
Case after the training
58.8%
  • Up to 11 days later
  • 43 to 52 days later
  • No weaker at the longer gap
Sellier, Scopelliti & Morewedge (2019), “Debiasing training improves decision making in the field,” Psychological Science 30(9). Bars run from 0 to 100%.

The idea

Why it works.

When you test a belief, the evidence that fits it is the evidence that comes to mind. Failures are memorable and get filed; the cases that went fine leave no trace. So the natural test of "the cold room ruins the bread" is to look at ruined bread, and that test can't come out against you.

The audit forces the other half of the comparison. Writing what you'd see if you were wrong turns a belief into a test. Looking at the cases that went fine gives the test a denominator. And sizing the audit to the stakes keeps it for the decisions where being wrong would cost you.

When it goes wrong

Habits that help.

  • Audit the few, not the many. If every lunch order gets a pre-mortem, the hiring decision gets a tired one.
  • Ask for the denominator. “How many times did we try it?” turns most alarming counts into rates.
  • Write your number first. Once you've heard someone else's estimate, yours drifts toward it. Put yours on paper before the meeting.
  • Don't reverse on one doubt. Over-correcting is a bias too. Change course when the comparison says so, not because the audit found something to worry about.

Go deeper

Keep going.

Practise this in HourLife.

In Decisions: Bias check. Write down a decision, weigh the options, mark the one you chose, and come back later to record how it turned out.

Download on the App Store Download on the Mac App Store How Decisions works in HourLife