Keep this life skill for later
Sign in to save
← All life skills

Learning & thinking · a life skill

Simple Experiment Design

flip one switch, and count

When people try to fix something, they change everything at once: new routine, new app, new bedtime. Then it works, or it doesn't, and they've learned nothing they can use again. A fair test changes one thing, keeps the rest the same, and counts something you wrote down before you looked.

  • Easy to start
  • Medium impact
  • Learn it in 1 week
  • One switch, a few nights
one switch, not four
The same question tested two ways: by flipping three things at once, and by flipping one and keeping the rest The question: will I read more before bed if my phone charges in the kitchen? On the left, three switches are flipped at once: phone in the kitchen, an earlier bedtime and a new book. Pages went up, but there's no telling which switch did it. On the right, only the phone switch is flipped and the bedtime, the book and the lamp are kept. Below, six nights: 6, 4, 7 pages as usual, then 12, 15, 11 with the phone out, medians 6 and 12, against a guess of 10 written down first. The card: change the phone's place; keep bedtime, book and lamp; count pages read, noted before sleep; guess 10 or more; check on the seventh day. The hunch, as a question If my phone charges in the kitchen, will I read more? Flip everything phone in kitchen earlier bedtime a new book the same lamp Pages went up. Which switch did it? ? Flip one phone in kitchen same bedtime the same book the same lamp Pages went up. One thing moved, so look there. Six nights, one switch guess 10 6 4 7 12 15 11 Mon Tue Wed Thu Fri Sat as usual · median 6 phone out · median 12 Change where the phone charges Keep bedtime, book, lamp Count pages read, noted before sleep Guess 10 or more a night Check Sunday, after three nights each 1 2 3 4 5

Both lanes got more reading. Only the right one can tell you why, because only one switch moved while the others stayed put.

Tonight, with a pen

Try it now.

One hunch about your own days: something you suspect would help if you changed it. Ten minutes to set up, a week to run.

  1. Write the hunch as a question

    "If I change this, will that go up or down?" A hunch you can't put that way isn't ready to test yet.

  2. Flip one switch

    List everything you're tempted to change, then pick one. The others can have their own week later.

  3. Write down what stays the same

    Bedtime, the book, the room. Naming them is what stops them drifting while you're not watching.

    If something can't stay the same, like the weather, write it down each day so you can see it later.

  4. Pick a number, and guess it first

    Pages, minutes, a 1 to 10 rating written at the same time each day. Then write the number you expect, before the first night.

  5. Run it, and set the day you'll check

    A few days as usual, then the same number of days with the switch flipped. Don't judge it until the check day.

Your own life is the lab. Coffee after lunch and your sleep, a walk before work and your mood, a list the night before and the time you start. Each one is a single switch and a number.

Your test tonight

List what you'd change, flip one, and write the card.

Put down everything you're tempted to change, one per line. Pick the one to test, and the rest move to the keep line by themselves. The checks under the card will tell you if your number is too vague to count, or if the test is too short to read.

    Your fair test card

    • Question
    • Change
    • Keep
    • Count
    • Guess
    • Check

      On the check day

      Type each day's number in, and the card compares the middle of each half with your guess.

      Fill in both halves to read the test.

      Nothing you type is saved or sent anywhere. It's gone when you leave the page, so copy the card and keep the numbers on paper by your bed.

      Why one switch

      Children who were taught why a fair test changes one thing learned to set one up, and used it on new problems. Being asked questions alone didn't teach it.

      Zhe Chen and David Klahr worked with children aged 7 to 10 on what scientists call the control of variables strategy: to find out what a thing does, change it and hold everything else the same. It's the rule most adults half know and rarely use on their own lives.

      Some children were taught it explicitly, with the reason it works, and then asked questions about the tests they'd set up. Others got the same questions with no teaching. The taught children learned to design fair tests and make sound inferences from them, and they carried the strategy over to other kinds of problem. The questions on their own didn't improve either.

      Age mattered for the longest leaps: the older children were better at using the strategy on problems that looked least like the ones they'd learned on.

      The honest limits: these were children working on classroom science problems, not adults testing their sleep. What carries over is the rule and the finding that knowing the reason helps you use it somewhere new. That's why the steps above ask you to write down what you're keeping the same, not just what you're changing.

      A fair test drawn as two ramps: the same ball, surface and length, and only the steepness differs same ball, same surface only the slope changes
      Shown why, then asked

      Explicit teaching of why a fair test changes one thing, plus questions about their own set-ups.

      Learned to design fair tests, and carried it to new kinds of problem.

      Only asked

      The same questions about their set-ups, with no teaching.

      Didn't get better at fair tests, or at the inferences drawn from them.

      Chen & Klahr (1999), Child Development. The ramps are our drawing of what a fair test looks like, not material from the study. The two cards summarise the comparison the abstract reports.

      The idea

      Why it works.

      A change you make by itself has nowhere to hide. If the number moves, you know where to look, and if it doesn't, you can cross that switch off and try the next one. Change four things and every result is a shrug.

      Writing the guess first matters too. Without it, whatever happens will feel like what you expected. With it, you find out how good your hunches are, and that's worth knowing for the next one.

      When it goes wrong

      Habits that help.

      • Count, don't feel. "Sleep better" can't be checked. Minutes awake, pages read or a 1 to 10 rating written at the same time each day can.
      • Write the number before you look back. Note it each day, then don't read the earlier days until the check. Knowing yesterday's number nudges today's.
      • Keep a weird-day note. A late night, a guest, a deadline. You won't remember them on the check day, and they explain the odd numbers.
      • Treat a short test as a hint. Three days each way can be fooled by one busy week. If it looks promising, run it again, and the next step up is to flip a coin for each day.

      Go deeper

      Keep going.

      Practise this in HourLife.

      In Experiments: Running experiments. Test one change for 7 to 30 days, rate it each evening, and record your own verdict when the run ends.

      Download on the App Store Download on the Mac App Store How Experiments works in HourLife