Archive for June, 2009
Does good experimental design require changing only one factor at a time (OFAT)?
Posted by mark in design of experiments, science on June 23, 2009
“Good experimental design usually requires that we change only one factor at a time” according to an article I read recently in The Scientist magazine (“Why Don’t We Share Data,” page 33, Issue 4, Volume 23). This guide for science fairs tells students that “you conduct a fair test by making sure that you change only one factor at a time while keeping all other conditions the same.”
Obviously changing two variables together makes no sense, such as the time that as science project one of my kids asked me to do a blind taste test on Coke versus Pepsi, but to keep them straight in their mind, she poured one cola in blue plastic cup and the other in white Styrofoam! Needless to say I was completely confounded.
The OFAT method is so engrained that it’s literally become the law according to scientist who told me that, when as an expert witness he presented statistically significant evidence, it was thrown out of court due to the experiment design having changed multiple factors simultaneously. What a crime!
Multifactor testing is far more effective for statistical power, screening efficiency and detection of interactions. Industrial experimenters are well-advised to forget their indoctrination in OFAT and make use of multifactorial designs. For reasons why, see my two-part series on Trimming the FAT out of Experimental Methods and No-FAT Multifactor Design of Experiments.
Good experimental design does NOT require changing only one factor at a time!
“Decisions taken by statistical professionals are final”
I’m just catching up on the Wall Street Journal issues that accumulated while I attended a statistical conference and then co-taught a workshop on Designed Experiments for Life Sciences. A June 3rd article by WSJs “Numbers Guy” Carl Bialik caught my eye with a graphic showing that most UK citizens distrust official statistics. This caused their government to create a Statistics Authority that will police other agencies on the numbers they release to the public. Here some key points as reported at this UK government web site:
So it seems that the number nerds will rule after all — just like they always dreamed when being belittled by the bullies who thought math and stats were simply a waste of time. Statisticians rule!
Rabid for numbered bones
I am absorbing a great deal of information from the 2009 American Statistical Association’s Quality & Productivity Research Conference at IBM’s Watson Research Center in Yorktown Heights, New York. However, since IBM sold off their PC business to the Chinese manufacturer Lenovo, I am not quite sure what’s being researched at this facility. The official word is that IBM now provides “solutions.” See if you can puzzle things out from this 2009 newsletter . But, for those who are hard-core ‘techies’, check out this impressive list of IBM R&D projects, which include such things as quantum mirages and blue genes.
IBM presents an impressive collection of calculating devices in the lobby of this R&D center. For example, see pictured an actual 1617 set of Napier’s bones made by the Scottish inventor of logarithms. Via a process called rabdology (from the Greek “rabid” for rod, and “logos” for calculating), these numbered rods of skeletal origin facilitated multiplication and the computation of square and cube roots.
PS. Coincidentally, I just saw the latest Star Trek movie. Being an engineer by profession, I am naturally drawn to the character Scotty. Having seen what his ancestor Napier did with bones (not to be confused with the Star Trek doctor “Bones”), I now understand why the Enterprise engineer is such a wizard. Given enough time, these Scots will solve any technical problem. “I canna change the laws of physics! I’ve got to have thirty minutes.“