Just the other day a new term came to light for me—a “bright line” rule. Evidently this is commonplace legal jargon that traces back to at least 1946 according to this language log. It refers to “a clear, simple, and objective standard which can be applied to judge a situation” by this USLegal.com definition.
I came across the term in this statement* on p-values from American Statistical Association (ASA) on statistical significance:
“Practices that reduce data analysis or scientific inference to mechanical ‘bright-line’ rules (such as ‘p < 0.05’) for justifying scientific claims or conclusions can lead to erroneous beliefs and poor decision-making.”
The ASA goes on to say:
“Researchers should bring many contextual factors into play to derive scientific inferences, including the design of the study, the quality of the measurements, the external evidence for the phenomenon under study, and the validity off assumptions that underlie the data analysis.”
It is hard to argue that if the p-value is high, the null will fly, that is, results cannot be deemed statistically significant. However, I’ve never bought into 0.05 being the bright-line rule. It is good to see ASA dulling down this overly simplistic statistical standard.
I can see the value for “bright line rules” in legal processes, a case in point being the requirement for the Miranda warning being given to advise US citizens of their rights when being arrested. However, it is ludicrous to apply such dogmatism to statistics.
*(The American Statistician, v70, #2, May 2016, p131)