Archive for October, 2011
Pareto distribution makes a handy ‘screw driver’
Posted by mark in Consumer behavior on October 30, 2011
“What scientific concept would improve everybody’s cognitive toolkit?” This was the 2011 question posed by sociologist Nicholas Christakis in his annual survey of science and technology gurus via the online salon Edge. Internet scholar Clay Shirky chose the Pareto distribution as a tool that could be put to better use for reducing income disparities. When I led manufacturing improvement teams, I graphed cumulative losses by cause and used this Pareto chart* to focus my engineering colleagues on the 20 percent that caused 80% of the problems. It seems that Christakis knew something when he put forward the Pareto early this year because, since then, the 99 Percent Project has taken aim at the 1 percent of folks from Wall Street who purportedly control all of our money. Being a technical type I am not interested in getting into issues of elitism. I’d rather experiment to identify the vital few factors that affect a system of interest.
Meanwhile, the United Nations declared this summer that tomorrow, October the 31st, the 7 billionth baby will be born. They may be overly precise on the timing, yet this is a very compelling statistic. Breaking down the world’s wealth by this population will continue to keep economists busy, but you can be sure it will maintain a “predictable imbalance” as observed by their colleague Pareto. Along those lines, I suggest you measure how Halloween candy gets distributed to your neighborhood population of trick-or-treaters. If a few big kids don’t take the lion’s share, I will eat my monster mask..
*The American Society of Quality (ASQ) provides a detailing of the Pareto Chart here.
Wrong more often than right but never in doubt
Posted by mark in Consumer behavior, Uncategorized on October 23, 2011
The New York Times Magazine provided a great readout on the “Surety of Fools”* in today’s issue. The author, psychologist Daniel Kahneman, starts by providing a real-life example of WYSIATI – “What you see is all there is.” Read his story to learn more about this, but basically it means that many times what you observe does not provide any meaningful information for predicting future behavior. Cocky Wall Street brokers are hit very hard, especially the males, who “act on their useless ideas significantly more often than women.” Ouch!
Kahneman examined the illusion of skill in a group of investment advisors who competed for annual performance bonuses. He found zero correlation on year-to-year rankings, thus the firm was simply rewarding luck. What I find most interesting is his observation that even when confronted with irrefutable evidence of misplaced confidence in one’s own ability to prognosticate, most people just carry on with the same level of self-assurance.
The bottom line is that you shouldn’t swallow everything said by assertive and confident people who advise on highly-variable systems such as financial markets. Heeding what an experienced physician suggests is one thing, but going with the most boastful money manager is another.
“True intuitive expertise is learned from prolonged experience with good feedback on mistakes.”
— Daniel Kahneman
“It’s what you learn after you know it all that counts.”
— John Wooden
* Posted earlier under a different title here.
Trouble with math & stats? Blame it on dyscalculia.
Posted by mark in Basic stats & math, Uncategorized on October 16, 2011
According to this article in Journal of Child Neurology “dyscalculia is a specific learning disability affecting the normal acquisition of arithmetic skills, which may stem from a brain-based disorder. Are people born with this inability to do math in particular, but otherwise mentally capable – for example in reading and writing? Up until now it’s been difficult to measure. For example, my wife, who has taught preschool for several decades, observes that some of her children progress much more slowly than other. However, she sees no differential in math versus reading – these attributes seem to be completely correlated. The true picture may finally emerge now that Michèle M. M. Mazzocco et al published this paper on how Preschoolers’ Precision of the Approximate Number System Predicts Later School Mathematics Performance.
Certainly many great minds, particularly authors, abhor math and stats, even though they many not suffer from dyscalculia (only numerophobia). The renowned essayist Hillaire Belloc said*
Statistics are the triumph of the quantitative method, and the quantitative method is the victory of sterility and death.
I wonder how he liked balancing his checkbook.
Meanwhile, public figures such as television newscasters and politicians, who appear to be intelligent otherwise (debatable!) say the silliest things when it comes to math and stats. For example a U.S. governor, speaking on his state’s pension fund said that “when they were set up, life expectancy was only 58, so hardly anyone lived long enough to get any money.”** One finds this figure of 58, the life expectancy of men in 1930 when Social Security began, cited often by pundits discussing the problems of retirement funds. Of course this was the life expectancy at birth, in times when infant mortality remained a much higher levels than today. According to this fact sheet by the Social Security Administration (SSA), 6.7 million Americans were aged 65 or older in 1930. This number exhibits an alarming increase. The SSA also provides interesting statistics on Average Remaining Life Expectancy for Those Surviving to Age 65, which show surprisingly slow gains over the decades. I leave it to those of you who are not numerophobic (nor a sufferer of dyscalculia) to reconcile these seemingly contradictory statistical tables.
*From “On Statistics”, The Silence of the Sea, Glendalough Press, 2008 (originally published 1941).
**From “Real world Economics / Errors in economics coverage spread misunderstandings” by Edward Lotterman.
Clickers allow students to vote on which answer is right for math questions
Posted by mark in Basic stats & math on October 12, 2011
Yesterday I attended a fun webinar on Interactive Statistics Education by Dale Berger of Claremont Graduate University. Because I was multitasking (aka “continuous partial attention” — ha ha) at work while attending this webinar my report provides just the highlights. However, you can figure out for yourself what they (the stats dept at Claremont) have to offer by going to this web page offering WISE (Web Interface for Statistics Education) tutorials and applets.*
After the presentation a number of educators brainstormed on interactive stats. David Lane of Rice U (author of many stat applets) suggested the use of “interactive clickers” – see this short (< 2 min.) newscast, for example. I wonder what happen when a majority vote for the wrong answer? For some teachers it might be easiest just to declare the most popular response as the correct answer. That would be consistent with the way things seem to be going in politics nowadays. ; )
*Just for fun try the Investigating the Central Limit Theorem (CLT) applet (click the link from the page referenced above or simply click here). This would be a good applet to provide when illustrating CLT using dice (such as is done in this in-class exercise developed by two professors from De Anza College). In this case, pick the uniform Population and sample size 2. Then Draw a Sample repeatedly, and, finally, just Draw 100 samples. Repeat this exercise with sample size 5 a la the game of Yahtzee (a favorite in my youth). Notice how as n goes up the distribution of averages becomes more normal and narrower. That’s the power of averaging.