A new book by David S. Salsburg provides a series of Cautionary Tales in Designed Experiments. Salsburg wrote the classic The Lady Tasting Tea, which I read with great delight. I passed along the titular story (quite amazing!) in a book review (article #4) for the July 2004 DOE FAQ Alert.
Salsburg’s cautionary tales offer a quick read with minimal mathematics on what can go wrong with poorly designed or badly managed experiments—mainly medical. I especially liked his story of the Lanarkshire Milk Experiment of 1930, which attempted to test whether pasteurization removed all the “good”. Another funny bit from Salsburg, also related in The Lady Tasting Tea and passed only by me in my review, stems from his time doing clinical research at Pfizer when a manager complained about him making too many “errors”. He changed this statistical term to “residuals” to make everyone happy.
With all the controversy now about clinical trials of Covid-19 vaccines and the associated politics, Cautionary Tales in Designed Experiments offers a welcome look with a light touch at how far science progressed over the past century in their experimental protocols.
“It is the well-designed randomized experiment that provides the final ‘proof’ of the finding. The terminology often differs from field to field. Atomic physicists look for “six sigma” deviations, structure-activity chemists look for a high percentage of variance accounted for, and medical scientists describe the “specificity” and “sensitivity” of measurements. But all of it starts with statistically based design of experiments.”
David S. Salsburg, conclusion to Cautionary Tales in Designed Experiments