Last week I attended a special webinar on “Statistical Significance and p-values” presented by the European Network of Business and Industrial Statistics (ENBIS). To my relief, none of the speakers called for abandoning the use of p values. Though I feel that p’s should not be a statistic to solely rely on for deeming results significant or not, when used properly they certainly reduce the risk of pressing ahead with spurious outcomes. It was great to get varying perspectives on this issue.
Here are a couple of fun quotes on that I gleaned from this ENBIS event:
- “Surely, God loves the .06 nearly as much as the .05. Can there be any doubt that God views the strength of evidence for or against the null as a fairly continuous function of the magnitude of p?” – Rosnow, R.L. & Rosenthal, R. “Statistical procedures and the justification of knowledge in psychological science”, American Psychologist, 44 (1989), 1276-1284.
- “My definition of a statistician is ‘one who prefers true doubts to false certainty’.” – Stephen Senn (Statistical Consultant, Edinburgh, Scotland, UK)
If you have a strong stomach for stats, see this Royal Society review article: The reign of the p-value is over: what alternative analyses could we employ to fill the power vacuum? It includes discussion of an alternative to p values called the “Akaike information criterion” (AIC). This interested me, because, as a measure for goodness of model-fit, Stat-Ease software provides AICc—a version of this statistic that corrects (hence the appendage “c”) for the small sample sizes of industrial experiments (relative to large retrospective scientific studies).