Tia Ghose of The Scientist provides a thought-provoking “Q&A” with biostatistician Peter Bacchetti on “Why small is beautiful” in her June 15th column seen here. Peter’s message is that you can learn from a small study even though it may not provide the holy grail of at least 80 percent power.* The rule-of-thumb I worked from as a process development engineer is not to put more than 25% of your budget into the first experiment, thus allowing the chance to adapt as you work through the project (or abandon it altogether). Furthermore, a good strategy of experimentation is to proceed in three stages:
- Screening the vital few factors (typically 20%) from the trivial many (80%)
- Characterizing main effects and interactions
- Optimizing (typically via response surface methods).
For a great overview of this “SCO” path for successful design of experiments (DOE) see this detailing on “Implementing Quality by Design” by Ronald D. Snee in Pharm Pro Magazine, March 23, 2010.
Of course, at the very end, one must not overlook one last step: confirmation and/or verification.
* I am loathe to abandon the 80% power “rule”** but, rather, increase the size of effect that you screen for in the first stage, that is, do not use too fine a mesh.
** For a primer on power in the context of industrial experimentation via two-level factorial design, see these webinar slides posted by Stat-Ease.