Individual Fish Risk Benefit Model

This page is set up as an adjunct to the discussion in The Science-Policy Shell Game concerning fish consumption advice.  I may replace the Excel macro linked here now with something prettier and/or easier to use at some point in the future.

Software

This Excel macro, Personal_Seafood_Net_Effect_Estimator.xls, integrates four components presented earlier:

Personal Fish Consumption Estimator

Methylmercury Toxicokinetic Conversions

Methylmercury Dose-Response

Fish Benefit Dose-Response

SPSG #13: Ending the Game

This chapter outlines strategies for overcoming the difficulties noted in earlier chapters. First, adopting some common legal strategies for separating Matters of Fact from Matters of Law could do wonders. The creation of job positions for Science Judges whose sole responsibility is to disentangle science matters from policy matters could facilitate that. Second, while eliminating the science-policy shell game entirely is probably not possible, there is no reason why it should be condoned or institutionalized. Therefore, the EPA assessment guidelines for cancer and noncancer endpoints both need to be rewritten. As the face of the Safety Assessment Paradigm, the Reference Dose background document should be rewritten to make it clear that the product of the assessment is a regulatory policy rather than a statement of scientific fact. As for the cancer risk assessment guidelines, instead of enshrining the default option with the Point of Departure, the guidelines should use probability trees to make the default option going away entirely. Furthermore, there is no reason to the restrict the use of quantitative risk assessment to just cancer. However, solving that problem will create another problem: At least in public health, a legacy of the institutionalized use of the science-policy shell game has virtually eliminated risk management as a federal job position. So, doing a risk assessment is of very little use if no one in the federal government has the responsibility for managing issues. That can happen, but position descriptions will need to be rewritten. Finally, research should be funded to support science, instead of supporting technocratic shell games. In particular, the enterprise of environmental epidemiology needs to be redesigned. Statistical significance testing should be eliminated as the primary means of drawing conclusions from data, and studies should be designed to increase or reduce evidentiary weight accorded to causal theories instead. Perhaps most importantly, observational data needs to be shared. When it comes to analyzing data, regardless of what their source of funding is, investigators cannot be given complete deference in conveying what the data infer. As an academic recommendation, teaching Statistics and Probability as separate subjects would clear up more than a few nagging philosophical problems.  Finally, the facade of impersonal scientific objectivity needs to be abandoned. Scientists should be both free to speculate and humble enough to admit their theories may be wrong.

SPSG #12: Personal Technique

This chapter is largely written in the first person, and it does so largely for the purpose of disparaging the concept of scientific objectivity. It starts out by describing how several reorganizations dramatically affected the branch at the USFDA that I worked in for 25 years. In the end, it was swallowed up by the shell game. It then goes on to discuss the importance of recognizing the subjective nature of science, particularly when the science is unsettled and uncertain. The objectivity facade is partly attributable to scientific writing style that takes the author out of discussions of factual issues, which hides tha fact that personal opnions are beign expressed.   To demonstrate what science is really like, I walk through the personal choices I made in developing the dose-response model for arsenic and lung cancer that was used for the apple juice and rice risk assessments discussed in Chapter 11. Some of those choices were done by committee and some were not, but either way they all involved subjective scientific judgements made in a fog of uncertainty. In one case, I made a different choice that I had previously because new information influenced my subjective judgment about how to go about estimating lifetime risks from a prospective epidemiological study, which underscores the notion that “objective” reality evolves with scientific inquiry. The resulting dose-response model is then used to provide risk estimates for someone with a high-end (for the United States) arsenic intake. In addition to providing the lifetime risk estimates that were also given in the FDA reports on apple juice and rice, estimated changes in average life expectancy are also provided. For the purpose of making an individual choice, the latter measure is far more meaningful. The chapter then suggests that the inability of EPA to provide a dose-response characterization for arsenic may stem from a wrongheaded demand for objectivity that dictates the use of the wrong probability and the wrong personnel for a job that needs statistical theory instead of statistical probability.

SPSG #11: The Technocracide

Since it kills the Safety Assessment Paradigm every time, it has always been very clear that arsenic would never make it as a food additive. Although arsenic commonly occurs in food as a contaminant, the concern for arsenic in food was always mitigated by the fact that the largest exposures have generally been from drinking water. That equation changed in 2001, when the EPA passed a regulation for arsenic in drinking water that changed that equation. Because the drinking water rule required a cost-benefit analysis, the decision process was supported by a risk assessment that produced risk estimates; in spite of the guidelines, it was consistent with the risk assessment paradigm. However, the Office of Water did find it necessary to hire outside consultants to accomplish that goal. In any case, arsenic exposure from water was reduced and arsenic in food became a relatively bigger issue as a result. As a result of public attention in 2011, the FDA issued guidance "action" levels for arsenic in apple juice in 2013 and rice in infant foods in 2016. From a risk management standpoint, both efforts were abject failures. Although risk assessments were produced, they didn't really support the guidance in either case. At least part of the reason is that setting levels usually isn't an effective way to manage risks from contaminants. Preventing something from getting into the food in the first place can be far easier, but that isn't always possible. Nonetheless, the FDA went ahead anyway with action levels anyway. In the case of apple juice, it isn't too hard to figure out why; the FDA commissioner publicly promised an action level before the risk assessment was done. The reasoning that went into the rice guidance is more mysterious. Even though there is nothing in the risk assessment to indicate that they are uniquely susceptible to arsenic, the FDA advised both infants and pregnant women to reduce their rice intake, but gave no advice for anyone else. In fact, the exposure assessment indicated that the greatest exposures to arsenic from rice are in adult males. On a more positive note, the FDA cancer risk assessments for both apple juice and rice solved the default option problem by using probability trees to represent the theoretical probability associated with the dose-response relationship for arsenic and both lung and bladder cancer. The main lesson to be learned from those exercises is that regardless of how well an assessment represents current science, if the message takes precedence over the result, there will be no reason to expect public health to improve.

SPSG #10: The Paradigm War

Methylmercury in fish has been a major issue for both the EPA and the FDA since several epidemics occurred in Japan and Iraq in the 60s and 70s. At about the same time (early 90s) as the FDA started quantifying risks for methylmercury in fish and issuing consumer advice for commercial seafood, the EPA started giving recreational fish consumption advice based on the EPA Reference Dose (RfD). In 1999 congress asked the National Academy of Science (NAS) to evaluate the RfD for methylmercury, and a report was issued in 2000. The fact that congress even asked the question of the NAS cemented in many people minds that the RfD was and is a statement of a scientific fact. That meant that if it was true for EPA then it had to be true for FDA too, and as a result any attempt to quantify the risks and provide information about what the risks came to be viewed as a political attempt to undercut the science. By 2004, the FDA and EPA had agreed to give joint advice for fish consumption, but there was no agreement about what the basis or the rationale for the fish advisory was. While the EPA thought the RfD was paramount, the FDA chose to pursue a quantitative strategy that balanced benefits and risks; so the fish-risk benefit assessment was an FDA-only affair. But perhaps the most important difference was about what information, if any, would be given to consumers. The RfD treats consumers in the same manner as it treats agency managers; it decides for them, and as a result there is no basis for providing the information to consumers that will let them decide for themselves. As an alternative, a few representative risk estimates are provided for the consumption of fish during pregnancy using a version of the risk assessment model developed for the FDA that is designed to estimate risks for individual consumers.

SPSG #9: A Practical Guide to Theoretical Probability

This Risk Analysis methodology chapter is the applied version of the philosophical discussion of probability presented in Chapter 2. It also fixes the flaws in the Redbook paradigm discussed in Chapter 4, resulting in the Guillotine paradigm. It begins with a discussion of characterizing uncertainty when there are both statistical and theoretical probabilities involved. While a theoretical probability does not need to be quantified when it is the only probability involved or when there is no decision at stake, giving it the same epistemic standing as a statistical one is unavoidable when both matter. However, that does not mean a theoretical probability can be used as if it were a statistical probability. A theoretical probability is perhaps true always or perhaps false always; is it not true sometimes and false at other times. The discussion then turns to the problem of assigning probabilities to alternative theories. Declaring that all sum to one is a simple matter, but deciding the probability of each theory is not. Since theoretical probabilities are subjective, depending on the opinions of those who have one (that usually means experts) is in some way is inevitable. However, instead of asking experts to assign probabilities to theories directly, there are advantage to garnering opinion in the form of evidential weights, where each alternative theory is evaluated more or less independently. Although formal weight-of-the-evidence schemes have been developed for many regulatory purposes, they are not usually thought of as quantitative exercises. However, it has been done and could be done better. It is also argued that WoE analysis and dose-response modeling need to be more tightly integrated, especially when the judgment that there is a causal relationship becomes more likely than not. First, the shape of the dose-response relationship may influence the judgment that there is a causal relationship. Second, the last vestiges of causal uncertainty may not matter if the estimated risks are too low to matter or high enough to be a concern even if they are only probable.

SPSG #8: The Wrong Probability

This chapter is about the problems associated with using statistical probability as the only probability, especially in epidemiology. While different scientific disciplines typically rely on somewhat different collections of convincing arguments, the conduct of epidemiology can aptly be compared to a trial for murder. Since the issue is causality, theoretical probability is front and center. Yet, at least when designing studies and publishing studies, environmental epidemiology studies often rely on tests of statistical significance testing for drawing conclusions. Arthur Bradford-Hill disparaged this practice over 50 years ago, and he is still quite right; statisticians are using the wrong probability. But that isn't the only problem. Epidemiologists (or their statisticians) often treat measures designed to quantify strength of association for the purpose of arguing causality as if they were measures of effect; thereby completely missing the point of having them in the first place. Next, epidemiologists are often reluctant to share raw data. While there are many possible explanations for this practice, the fact that other analysts would be able to use the data to explore and support theories not utilized in the published report is chief among them. The data sharing problem becomes especially evident when the theories used in published analyses are obviously wrong, either when they are first published, or perhaps later. This more or less forces the court of scientific opinion to rely on hearsay evidence. As an example of that, the use of log transformed measures of dose in multivariate regression analyses is discussed. Since it is an established analytical procedure, there is a tendency to think of regression analysis as a "theory-free" analysis that provides conclusions that are largely empirical. But, that isn't true at all. Linear regression analysis presumes that the quantitative dose-response relationship is linear. Similarly, doing a linear regression analysis with the log of dose presumes that the quantitative dose-response relationship is loglinear. But that results in a supralinear function where not only do the effects get bigger as the dose gets smaller, the effect approaches infinity as the dose approaches zero. Even though that's quite impossible, the practice continues, and that is probably because testing a theory that is definitely wrong is a reliable way of producing scary statistically significant low dose effects.