In our business we focus a lot of energy to convince experimenters they must conduct enough runs to develop the statistical power needed for detecting an effect of interest. What amazed me about the recent discovery of the Higgs boson is the sample size required to see this “strange pink elephant” as it’s described in the embedded explanatory video cartoon. The boffins of CERN took 40 million measurements per second for 20 years. These physics fellows cannot be topped for being persistent, tenacious, dogged and determined. Good for them and, I suppose, us.
“If the particle doesn’t exist, one in 3.5 million is the chance an experiment like the one announced would nevertheless come up with a result appearing to confirm it does exist.”
– Carl Bialik, ‘The Numbers Guy’ for Wall Street Journal explaining in his July 7-8 column the statistical meaning of CERN’s 5 sigma standard of certainty (see How to Be Sure You’ve Found a Higgs Boson).
#1 by Eric Kvaalen on August 26, 2012 - 1:39 pm
It’s amazing how often you see the wrong explanation of what 5 sigmas means, even from science magazines like New Scientist. They say it means that there’s a chance of 1 in so many million THAT the result is a fluke. Your Carl Bialik got it right.