In the latest issue of American Scientist magazine, Howard Wainer, an adjunct professor at Wharton, makes a case for how ignorance of how sample size affects statistical variation has created havoc for nearly a millennium – and continues to do so today. Simply put, the variance of sample means increase as the sample size decreases. Wainer deems Abraham de Moivre’s 1730 discovery of this mathematical relation The Most Dangerous Equation .
The article details a major investment by Bill and Melinda Gates Foundation and others to provide better education via smaller schools. Statistics showed that among high-performing schools were an unrepresentatively large proportion of smaller ones. However, the Wainer found that the same can be said for low-performing schools – smaller ones are over-represented. This finding agrees with De Moivre’s equation – smaller schools display greater variance of test score averages and thus over-populate the extremes. Wainer presents statistical evidence that “overall bigger schools do better…not unexpected, since very small high schools cannot provide as broad a curriculum or as many specialized teachers.” He concludes that “Spending more than a billion dollars on a theory based on ignorance of de Moivre’s equation – in effect serving only to increase variation – suggests just how dangerous that ignorance can be.”
Wainer provides another example showing how U.S. cities rank for automobile safety. Smaller cities come out on top and bottom – again demonstrating how variance increases as sample size decreases. How often have you seen tables like this in the popular media that rank areas by some criterion and seen the same result – smaller cities coming out on top and bottom? Perhaps some of the investments for improving education ought to be directed toward basic statistics!