Archive for March, 2010
Misuse of statistics calls into question the credibility of science
Posted by mark in science, Uncategorized on March 28, 2010
The current issue of Science News features an indictment of statistics by writer Tom Siegfried. He pulls no punches with statements like this:
“…a mutant form of math has deflected science’s heart..”
“Science was seduced by statistics…”
“…widespread misuse of statistical methods makes science more like a crapshoot.”
“It’s science’s dirtiest secret: …testing hypotheses by statistical analysis stands on a flimsy foundation.”
“Even when performed correctly, statistical tests are widely misunderstood and frequently misinterpreted. As a result, countless conclusions in the scientific literature are erroneous…”
Draw your own conclusions on whether science fails to face the shortcomings of statistics by reading Siegried’s article Odds Are, It’s Wrong.
My take on all this is that the misleading results boil down to several primary mistakes:
- Confusing correlation with causation
- Extrapolating from the region of experimentation to unstudied areas
- Touting statistically significant results that have no practical importance
- Reporting insignificant results from studies that lack power to see differences that could be very important as a practical matter.
I do not think statistics itself should be blamed. A poor workman blames his tools.
Test and evaluation of the Great Panjandrum – a spectacular failure for weaponry
Posted by mark in history, science, Uncategorized on March 23, 2010
When time becomes available – mainly while I do cardio-exercise on my home elliptical, I’ve been watching a classic 26-episode BBC series on The World at War that my oldest son gave me. It’s extremely compelling – rated 9.7 out of 10 by over two thousand voters at the Internet Movie Database (IMDb).
This morning I watched the chronicle of D-Day. Being that I just returned from the Annual National Test & Evaluation Conference by the National Defense Industry Association (NDIA), it was interesting to see what the boffins of Britain invented to defeat the defenses put up along the Normandy beaches. Perhaps the most amazing device was the Great Panjandrum, a rocket-propelled cart, which according to this write-up for Wikipedia was developed by the Admiralty’s Directorate of Miscellaneous Weapons Development. (One wonders about the “miscellaneous” bit.) The clip I viewed on the spectacular failure of the Great Panjandrum can be seen (along with other incredibly-inept military devices for D-Day) in a video on British Secret Wartime Follies posted in this article by UK’s Daily Mail. Check it out!
PS. The Brits continue to come up with the most audacious inventions, such as this flame-throwing moped developed as a deterrent against derelict drivers competing for motorway lanes.
Evolutionary operation
Posted by mark in design of experiments, Uncategorized on March 7, 2010
Last December, after an outing by the Florida sea, I put out an alert about monster lobsters. This reminded me of an illustration by statistical gurus Box and Draper* of a manufacturing improvement method called evolutionary operation (EVOP), which calls for an ongoing series of two-level factorial designs that illuminate a path to more desirable conditions.
With the aid of Design-Expert® software, I reproduced in color the contour plot in Figure 1.3 from the book on EVOP by Box and Draper (see figure at the right). To illustrate the basic principle of evolution, Box and Draper supposed that a series of mutations induced variation in length of lobster claws as well as the pressure the creatures could apply. The contours display the percentage of lobsters at any given combination of length and pressure who survive long enough to reproduce. Naturally this species then evolves toward the optimum of these two attributes as I’ve shown in the middle graph (black and white contours with lobsters crawling all over them).
In this way, Box and Draper present the two key components of natural selection:
- Variation
- An environment that favors select variants.
The strategy of EVOP mimics this process for improvement, but in a controlled fashion. As illustrated here in the left-most plot, a two-level factorial,** with ranges restricted so as not to upset manufacturing, is run repeatedly – often enough to detect a significant improvement. In this case, three cycles suffices to power up the signal-to-noise ratio. This case illustrates a big manufacturing-yield improvement over the course of an EVOP. However, any number of system attributes can be accounted for via multiple-response optimization tools provided by Design-Expert or the like. This ensures that an EVOP will produce more desirable operating conditions overall for process efficiency and product quality.
It pays to pay attention to nature!
*Box, G. E. P. and N. R. Draper, Evolutionary Operation, Wiley New York, 1969. (Wiley Classics Library, paperback edition, 1998.)
**(We show designs with center points as a check for curvature.)
Beware of bugs bearing backpacks
Posted by mark in science, Uncategorized on March 3, 2010
I am attending a conference sponsored by the National Defense Industry Association (NDIA). They provided all of us participants a copy of the latest issue (March) of their publication National Defense. While wiling away the time listening to some long-winded higher-ups I paged through the magazine and admired the weaponry developed to keep our war-fighters supported to the max. However, on page 17 a very odd picture caught my eye – a cockroach carrying a radiation sensor on its back! A researcher at Texas A&M reports that these bugs are ideal for sweeping potentially contaminated areas, ideally in teams of twenty. They can be operated remotely via devices that stimulate their leg muscles.
There is one problem though: Cockroaches cannot crawl backward. One had better hope that none of the bad guys wear pointy-toed cowboy boots, because they will be ideal for killing the sensor-bearing bugs that become stuck in the corners.