SPSG #8: The Wrong Probability

This chapter is about the problems associated with using statistical probability as the only probability, especially in epidemiology. While different scientific disciplines typically rely on somewhat different collections of convincing arguments, the conduct of epidemiology can aptly be compared to a trial for murder. Since the issue is causality, theoretical probability is front and center. Yet, at least when designing studies and publishing studies, environmental epidemiology studies often rely on tests of statistical significance testing for drawing conclusions. Arthur Bradford-Hill disparaged this practice over 50 years ago, and he is still quite right; statisticians are using the wrong probability. But that isn't the only problem. Epidemiologists (or their statisticians) often treat measures designed to quantify strength of association for the purpose of arguing causality as if they were measures of effect; thereby completely missing the point of having them in the first place. Next, epidemiologists are often reluctant to share raw data. While there are many possible explanations for this practice, the fact that other analysts would be able to use the data to explore and support theories not utilized in the published report is chief among them. The data sharing problem becomes especially evident when the theories used in published analyses are obviously wrong, either when they are first published, or perhaps later. This more or less forces the court of scientific opinion to rely on hearsay evidence. As an example of that, the use of log transformed measures of dose in multivariate regression analyses is discussed. Since it is an established analytical procedure, there is a tendency to think of regression analysis as a "theory-free" analysis that provides conclusions that are largely empirical. But, that isn't true at all. Linear regression analysis presumes that the quantitative dose-response relationship is linear. Similarly, doing a linear regression analysis with the log of dose presumes that the quantitative dose-response relationship is loglinear. But that results in a supralinear function where not only do the effects get bigger as the dose gets smaller, the effect approaches infinity as the dose approaches zero. Even though that's quite impossible, the practice continues, and that is probably because testing a theory that is definitely wrong is a reliable way of producing scary statistically significant low dose effects.