Less is More
The 1986 Cancer Risk Assessment Guidelines were only about
20 pages long, giving only a broad outline about how to proceed. In over 200 pages, the 2005 Cancer Risk
Assessment Guidelines describe a far more codified process that doesn’t result
in a risk estimate at all. As an interim
replacement guideline, this is much better:
Thou shalt estimate
the risk.
Sure, some more general guidance could be proffered, but 20
pages should be plenty. Besides leaving
Points of Departures in the dust, another concept that can be retired is the
idea that quantitative risk assessment is just for cancer. If a potential health effect is worth caring
about, it is worth quantifying. It does
not matter whether the endpoint is cancer or not.
Usually, the hardest part of producing a public health risk
assessment is the dose-response analysis.
With a little more thought and some better data, it can always be done
better. For that reason alone,
prescriptive guidelines are a bad idea. But,
here is a useful guideline:
Keep it simple,
improve as necessary.
Risk assessment isn’t an academic exercise. It won’t produce any great lasting scientific
truths. At best, it will distill and
make use of current knowledge for the purpose of informing current policy
decisions. Answer the question first,
then work on a better answer after that. The EPA Benchmark Dose modeling program is
good for starters; instead of the benchmark dose estimates, make use of the
model parameter estimates.
How good the risk assessment or the dose-response analysis
needs to be can vary widely depending on the decision at stake. Trying to do it perfectly the first time is a
dumb idea. It will never be perfect
anyway, and it probably doesn’t need to be.
It doesn't have to done all at once, either. You can estimate the risk for one health end point now, and get around to another one later. Risk assessment is an iterative
process where some revision may be necessary every time it is submitted to peer
reviewers or the public for comment. The
only thing that stops it is a final decision.
Uncertainty Analysis
Just about every one of the many treatises on public health risk
assessment written over the last 30 years has paid homage to the importance of
characterizing uncertainty. The first
reason for that is there is usually lots of it.
The second reason is that without a credible characterization of the
range of plausible interpretation it is virtually impossible to produce a risk
estimate that isn’t politically biased in some way. But that advice has not been fully heeded, so
a little extra guidance just might spur things on a bit:
- The Uncertainty Analysis IS the Risk Assessment. It is not something to be tacked on later. In fact, a good way to proceed to start by producing a range of how big the risks might be, and then work on filling in the probability distribution in between after that. Even if it is not possible to include every conceivable source of uncertainty, the important ones should be, because otherwise they won’t count. Sensitivity analyses that offload the uncertainty into a separate analysis that doesn’t figure into the decision don’t really count either.
- It’s Not Just Statistics. The general perception of probability is that it is just one of the many flavors of statistics, and therefore the way to characterize uncertainty is to hand the data off to a statistician so the uncertainty can be quantified. In fact, some of the uncertainty can be represented that way, but it quite often happens that the major uncertainties are something else entirely.
- Theoretical Probability. The other great source of uncertainty arises when there are two or more theories that may be used to explain the data or describe reality. The shape of the dose-response function is the most common occurrence of this type of uncertainty in public health, but there can be other instances of it as well. Since it is a product of scientific reasoning, the main responsibility for characterizing this type of uncertainty has to lie with a scientist rather than a statistician. Theoretical probabilities can be represented with probability trees which involves giving each competing theory a probability that is based the relative “weight of the evidence” for each theory. While tree probabilities are not statistical or even mathematical, if probabilities are assigned so that they sum to one, theoretical probabilities can be mixed and matched with statistical probabilities.
Problem and Solution Formulation
The latest treatise on risk assessment from the National
Academy of Sciences (2009) emphasizes the importance of identifying the
regulatory issues that a risk assessment needs to address. If the risk assessment is really intended to
provide useful information, this should not be difficult. Every formal risk assessment is preceded by a
subjective one, so regulatory issues and a rough idea of the likely answer should
be known before it begins.
It may be even more important to identify how the problem
might be solved. If there is a significant
risk, how can it be eliminated or reduced?
The risk assessment may then be designed to evaluate how effective
different regulatory intervention efforts may be.
Monte-Carlo Simulation
Even though the 2005 Guidelines contain some very useful
discussion of weight of the evidence evaluations, the guidelines conclude with
instructions for writing a risk characterization “narrative”. If the intent were to guide the production of
a risk assessment, instead of trying to prevent one, that wouldn’t be necessary. What you really need for a risk characterization
is a computer programmer. The exposure
and dose-response analyses will produce two or more model bits that need to be put
together to obtain an estimate of which will have some associated uncertainty. You can calculate a worst case, best case, and
a most likely estimate without a computer, but a Monte-Carlo simulation or
something like will give a more complete picture of what is currently known.
Since it is a public health risk assessment, a one
dimensional simulation may not be enough.
You can produce an estimate of the frequency of disease in a population where
the only distribution is uncertainty.
But, a two-dimensional simulation where one dimension describes
variability in the population, while the other characterizes the uncertainty is
far more informative. Transforming
disease frequency measures into life expectancy or some other measure that can
be interpreted as an effect of an individual can help make this happen.
In any case, after the calculations are complete, the
estimates may be tabulated, graphed, and explained. A narrative should not be necessary. However, it is not unlikely that there will more
questions that need to be answered.
References
National Research Council (2009). Science
and Decisions: Advancing Risk Assessment. National Academy of Sciences Press, Washington, DC.
USEPA (1986). Guidelines
for Carcinogen Risk Assessment. EPA/630/R-00/004
USEPA (2005). Guidelines
for Carcinogen Risk Assessment. EPA/630/P-03/001F.
Official Post Soundtrack
Post Notes
Thesis Post #43. Disclaimer: This is not an official EPA publication. Do cite, quote, or plagiarize.
No comments:
Post a Comment