In the early 1990s I spoke at an applied statistics conference attended by DOE gurus George Box and Stu Hunter. This was a time when Taguchi methods had taken hold, which engineers liked because the designs eschewed randomization in favor of ordering by convenience–with hardest-to-control factors changed only once during the experiment. I might have fallen for this as well, but in my early days in R&D I worked on a high-pressure hydrogenation unit that, due to risks of catastrophic explosion, had to be operated outdoors and well away from any other employees. (Being only a summer engineer it seemed that I was disposable.) Naturally the ambient conditions varied quite dramatically at times, particularly in the Fall season when I was under pressure (ha ha) to wrap up my project. Randomization of my experiment designs provided me insurance against the time-related lurking variables of temperate, humidity and wind.
I was trained to make runs at random and never questioned its importance. Thus I was really surprised when Taguchi disciples attending my talk picked on me for bothering to do so. But, thank goodness, Box had already addressed this in his 1989 report Must We Randomize Our Experiment. He advised that experimenters:
- Always randomize in those cases where it creates little inconvenience.
- When an experiment becomes impossible being subjected to randomization
- and you can safely assume your process is stable, that is, any chance-variations will be small compared to factor effects, then run it as you can in non-random order;
- but, if due to process variation, the results would be “useless and misleading” without randomization, abandon it and first work on stabilizing the process;
- or consider a split-plot design.
I am happy to say that Stat-Ease with the release of version 9 of its DOE programs now provides the tool for the compromise, as Box deems it, between randomizing or not, that is—split plots. For now it is geared to factorial designs, but that covers a lot of ground for dealing with hard-to-change factors such as oven temperature in a baking experiment.* Details on v9 Design-Expert® software can be found here http://www.statease.com/dx9.html along with a link to a 45-day free trial. Check it out!
*For a case study on a split-plot experiment that can be easily designed, assessed for power and readily analyzed with the newest version of Stat-Ease software, see this report by Bisgaard, et al (colleagues of Box).