Posts Tagged science
Picking on P in these times of measles
Posted by mark in science, Uncategorized on February 3, 2015
Randall Munroe takes a poke at over-valuers of p in this XKCD cartoon
Nature weighed in with their shots against scientists who misuse P values in this February 2014 article by statistics professor Regina Nuzzo. She bemoans the data dredgers who come up with attention-getting counterintuitive results using the widely-accepted 0.05 P filter on long-shot hypotheses. A prime example is the finding by three University of Virginia finding that moderates literally perceived the shades of gray more accurately than extremists on the left and right (P=0.01). As they admirably admitted in this follow up report on Restructuring Incentives and Practices to Promote Truth Over Publishability, this controversial effect evaporated upon replication. This chart on probable cause reveals that these significance chasers produce results with a false-positive rate of near 90%!
Nuzzo lays out a number of proposals to put a damper on overly-confident reports on purported scientific studies. I like the preregistered replication standard developed by Andrew Gelman of Columbia University, which he noted in this article on The Statistical Crisis in Science in the November-December issue of American Scientist. This leaves scientists free to pursue potential breakthroughs at early stages when data remain sketchy, while subjecting them to rigorous standards further on—prior to publication.
“The irony is that when UK statistician Ronald Fisher introduced the P value in the 1920s, he did not mean it to be a definitive test. He intended it simply as an informal way to judge whether evidence was significant in the old-fashioned sense: worthy of a second look.”
The best accidental inventions of all time
I learned from my latest issue of Chemical and Engineering News that Stanley Stookey of Corning Glass Works died last month at age 99. In 1952 he mistakenly heated an alumino-silicate glass to 900 degrees C meaning only to top out at 600. After much cursing, according to the CEN story, Stookey found that instead of the molten mess expected, the material crystallized into a new type of material called a glass ceramic that proved to be “harder than carbon steel yet lighter than aluminum—shatterproof.”
Being in the business of planned experimentation it always amazes me to come across stories like this of serendipitous science. Obviously chance favors the prepared mind because most of the momentous discoveries are made by world-class chemists such as Stookey and others of his kind in the fields of physics and so forth.
I am a huge fan of 3M Post-It® Notes, not only due to their incredible usefulness, but also because it delights me to think of my fellow Minnesotan Art Fry coming up these by accident. For a list including him and a dozen other experts in their field who made the most of mishaps into inventions see 13 Accidental Inventions That Changed The World by Drake Baer of Business Insider. The one I like best is George Crum (great surname for a chef!) who reacted to customer complaining about his French fries by slicing them into ridiculously thin and hard-backed pieces. Never mind that it probably was his sister Katie who made the accidental discovery according to this Snopes investigation. Either way this works out to be a delicious story.
My advice to our clients is to keep a close watch for any strange results that crop up as statistically deviant in the course of a designed experiment. They may turn out to be really Crummy!
Japanese tribologists confirm that banana peels are slippery
We all know that banana peels are slippery, but who suspected this would be worthy of study. I suppose that given there’s a field of study (tribology) that focuses exclusively on the rubbing of surfaces, it stands to reason that the friction of fruits would come under scrutiny. The Japanese researchers who studied banana peels were singled out for an IgNobel prize for an achievement that made people “LAUGH, and then THINK.” It turns out that learning what makes these fruit coverings (aka “epicarps”) so slippery might lead to better lubrication of artificial joints.
It really is amazing how much a banana peel does to reduce friction. Check out the data shown in this report by Business Insider. As noted here in a Science News blog a floor littered with peels is not good for monkeying around, being nearly as slippery as ice, which us Minnesotans can readily appreciate. It would be funny, though, to see someone try this. Ha ha—someone already did as you can see in this Mythbuster’s clip of slips.
The rarest of birds—a reproducible result from a scientific study
Posted by mark in science, Uncategorized on February 3, 2014
In The New York Times new column Raw Data, science writer George Johnson laments experimenters
“ways of unknowingly smuggling one’s expectations into the results, like a message coaxed from a Ouija board.”
– Science Times, 1/21/14
This, of course, leads to irreproducible findings.
As a case in point, only 6 of 53 landmark papers about cancer found support in follow up studies, even with the help of the original scientists working in their own labs, according to an article in the Challenges in Irreproducible Research archive of Nature cited by Johnson.
That is discouraging but I am not surprised. I feel fairly sure that the any assertions of import get filtered very rigorously until only ones that reproduce reliably make it through.
The trick is to remain extremely skeptical of initial reports, especially those that get trumpeted and reverberate around the popular press and the internet. Evidently it is human nature to then presume that when an assertion is repeated often enough then it must be true, even though it has not yet been reproduced. Saying it’s so does not make it so.
If you squint while taking a test will it improve your score?
In a 2007 experiment, researchers at Princeton split 40 math students into two groups for a test written up in two fonts—one clear and the other difficult (italic, light gray). Counter-intuitively the latter group scored 29 percent higher. In his new book David and Goliath, Malcolm Gladwell, renowned for the best seller the popular Outliers, cites this as an example of how “facing overwhelming odds produces greatness,” or, as Nietzsche said “That which does not kill us makes us stronger.” This works for me—being up for challenges, but perhaps it cannot be extrapolated to the people in general. As Christopher Chabris cautions in this Wall Street Journal review yestertoday, provocative results like the ones from Princeton often cannot be reproduced. He warns:
“Anyone who has followed recent developments in social science should know that small studies with startling effects must be viewed skeptically until their results are verified on a broader scale. They might hold up, but there is a good chance they will turn out to be spurious.”
If it seems too counter-intuitive to be true, perhaps it isn’t—best in these cases you await confirmation by others in adequately-powered verification experiments.
Kids & Science
Posted by mark in pop, science, Uncategorized on June 23, 2013
I am heartened to hear of great work being done by current and former colleagues to get K-12 kids involved in Science, Technology, Engineering, and Mathematics (STEM). For example, Columbia Academy, a middle school (grades 6-8) in Columbia Heights (just north of Minneapolis), held an Engineering and Science Fair last month where two of our consultants, Pat Whitcomb and Brooks Henderson, joined a score of other professional engineers who reviewed student projects. Winners will present their projects this summer at the University of Minnesota’s STEM Colloquium.
Also, I ran across a fellow I worked with at General Mills years ago who volunteers his time to teach middle-schoolers around the Twin Cities an appreciation for chemistry. He makes use of the American Chemical Society (ACS) “Kids & Chemistry” program, which offers complete instructions and worksheets for many great experiments at middle-school level. Follow this link to discover:
– Chemistry’s Rainbow: “Interpret color changes like a scientist as you create acid and base solutions, neutralize them, and observe a colorful chemical reaction.”
– Jiggle Gels: “Measure with purpose and cause exciting physical changes as you investigate the baby diaper polymer,* place a super-absorbing dinosaur toy in water, and make slime.”
– What’s New, CO2? “Combine chemicals and explore the invisible gas produced to discover how self-inflating balloons work.”
– Several other intriguing activities contributed by ACS members.
Kudos to all scientists, engineers, mathematician/statisticians who are engaging kids in STEM!
*(The super-slurpers invented by the diaper chemists really are quite amazing as I’ve learned from semi-quantitative measurements of weight before and after soakings by my grandson. Thank goodness! Check out this video by “Professor Bunsen”, which includes a trick to recover the liquids that I am not going to try.)
A strange pink elephant — the Higgs boson
Posted by hank in science, Uncategorized on July 9, 2012
In our business we focus a lot of energy to convince experimenters they must conduct enough runs to develop the statistical power needed for detecting an effect of interest. What amazed me about the recent discovery of the Higgs boson is the sample size required to see this “strange pink elephant” as it’s described in the embedded explanatory video cartoon. The boffins of CERN took 40 million measurements per second for 20 years. These physics fellows cannot be topped for being persistent, tenacious, dogged and determined. Good for them and, I suppose, us.
“If the particle doesn’t exist, one in 3.5 million is the chance an experiment like the one announced would nevertheless come up with a result appearing to confirm it does exist.”
– Carl Bialik, ‘The Numbers Guy’ for Wall Street Journal explaining in his July 7-8 column the statistical meaning of CERN’s 5 sigma standard of certainty (see How to Be Sure You’ve Found a Higgs Boson).
Ivory towers of academia (& shiny ones in Vancouver)
Today’s Vancouver Sun suggests that a competitive university culture discourages sharing of knowledge, which then leads to the publication of many flawed and fraudulent studies. This is a rehash of issues I cited recently with the warning Beware of obvious answers and positive results. It would be great, albeit a bit boring, if journals published negative results from well-designed experiments with adequate power to see beneficial results. As my colleague Wayne Adams says
“Most of what you learn from an experiment is what NOT to do.”
PS. I took this picture Granville Island looking across to downtown Vancouver.
Marshmallows measure the speed of light (and get put to other good uses)
One of my favorite blogs, Flowing Data, provided me the heads-up on a great lecture by Adam Savage (the Mythbuster’s guy) called “Simple ideas lead to scientific discoveries”. I really enjoyed all of his stories, but especially the one on Hippolyte Fizeau’s measurement of the speed of light in 1849. Ingenious!
Coincidentally, my brother Paul forwarded me a detailing of how one can measure the speed of light with a tray of mini-marshmallows! Check it out at this Science Blog written by theoretical astrophysicist Ethan Siegel. This sharp-fingered fellow (if you view his blog you will see what I mean) goes on to tout a marshmallow-made diorama that ‘peeps’ recent claims of particles going faster than the speed of light.
That leads me to puffing up my daughter Emily, who achieved the “peeple’s choice” award in the Saint Paul Pioneer Press Ninth Annual Marshmallow Peeps Diorama Contest. She and two of her closest peeps produced The Mupeeps Take Minnepeepolis. It looks very much like the view out of my window from Stat-Ease headquarters east of downtown Minneapolis.
By the way, my favorite Muppets are Bunsen and Beaker. See them demo their invention of fireproof paper here. At the Muppets Lab one should always be prepared with fresh marshmallows on a stick. I advise going for two at a time. o——<8
Speed of light exceeded (astounding!)? Or was it measurement error?
This morning I read this NY Times news that European physicists measured neutrinos at 0.0025 percent above the speed of light. If so, it may be only a matter of time before you can send yourself a telegram to not do whatever you did that you’ve always regretted and, by the way, to please invest a thousand dollars in Microsoft, Facebook or the like (depending on the timing).
Years ago I visited Mount Wilson Observatory in California with my son Hank. See me pictured by their two domes that house 60 and 100 inch telescopes; respectively. This was the center for landmark experiments on the speed of light as detailed in this Wikipedia article. Obviously measurement error made this a very difficult.
Being a skeptic, and seeing that a similar experiment* found neutrinos whizzing about at the speed of light, but not beyond that, I was going to advise caution. However, Hank gave me the heads up to today’s xkcd cartoon (click the image to make it bigger and more readable). I think this guy has got a better idea.
*Done with a group at the Soudan Underground Laboratory here in Minnesota. They first did physics experiments there, in an abandoned iron mine, in 1980. I featured this in a retro young-adult techno/adventure/mystery/thriller called The Secret of the Wolf Ring (Amazon, Kindle Edition).