Knowing of my interest in experiment design,my son-in-law (a newly minted PhD chemist) showed me his book on Cooking for Geeks. It offers a lot of fun detail on chemistry for a fellow like him. As a chemical engineer by profession, I like that too. Furthermore, I am all for the author’s enthusiasm for experimentation. However, his methodology, quoted below, lacks any sophistication or statistical power.
Make a recipe twice, changing just one thing (cookies: melt the butter or not?), and see what changes (if anything). If you’re not sure which way to do something, try both and see what happens. You’re guaranteed to learn something—possibly something the recipe writer didn’t even understand.
– Jeff Potter, author of Cooking for Geeks
Potter is not alone in remaining mired in OFAT and sample sizes of one (n1). This is also the methodology of the prestigious Cooks Illustrated as seen by this experiment on roasting ribs. Chris Kimball who launched this magazine, and, until recently, hosted “Americas Test Kitchen” on PBS, contacted me soon after Forbes recommended Stat-Ease software for multivariable testing (MVT) in March of 1996 (“The New Mantra: MVT”. I gave him a briefing on multifactor (as I prefer to deem it) design of experiments. However, Chris told me that his cooks were artists, not scientists, and they would not take to anything other than n1 OFAT. That works only when you make gross changes, such as roasting at 250 versus 450 degrees F. Even then, I’d like to see at least 4 of each level done in a randomized plan, and, better yet, a multifactor experiment.
The one nice thing about these poorly executed food experiments is that you can re-do them yourself. I might take on the question of roasting ribs, for example. Yum!