Stats Made Easy

In a 2007 experiment, researchers at Princeton split 40 math students into two groups for a test written up in two fonts—one clear and the other difficult (italic, light gray).  Counter-intuitively the latter group scored 29 percent higher.  In his new book David and Goliath, Malcolm Gladwell, renowned for the best seller the popular Outliers, cites this as an example of how “facing overwhelming odds produces greatness,” or, as Nietzsche said “That which does not kill us makes us stronger.”  This works for me—being up for challenges, but perhaps it cannot be extrapolated to the people in general.  As Christopher Chabris cautions in this Wall Street Journal review yestertoday, provocative results like the ones from Princeton often cannot be reproduced.  He warns:

“Anyone who has followed recent developments in social science should know that small studies with startling effects must be viewed skeptically until their results are verified on a broader scale.  They might hold up, but there is a good chance they will turn out to be spurious.”

If it seems too counter-intuitive to be true, perhaps it isn’t—best in these cases you await confirmation by others in adequately-powered verification experiments.