Probability is the Guide of Life
For personal decisions, theoretical uncertainty is the far
more familiar form of probability. If
two different sources of information lead to different courses of action, then
you have to either decide who and what to trust or hedge your bets. However, the probability of chance that is
amenable to a mathematical treatment and is the main form found in academic discourse can be important too. The
relative importance of the two probabilities can vary with the problem. Sometimes one or the other will dominate,
while in other instances both are important.
Recreational betting games serve as an example:
- Roulette. Betting on a roulette wheel is purely a game of chance. The odds and a long term expected return can be calculated very accurately. Well, unless the game is fixed.
- Horse Racing. In theory, some horses are faster than others – chance has very little to do with who wins. Sure, historical records are important, but that’s mainly because they indicate which horses are fast and which ones are not.
- Poker. The odds that a certain card or cards will turn up can be calculated, and the game of poker can be simply played as a game of chance. But good poker players also take the mannerisms of their opponents into account when they bet, which turns poker into a mixed probability game.
It’s the last category of problems that make risk analysis
interesting.
Betting on the Single Instance
If you are betting on a single instance (i.e. what to do
now), then boiling down theoretical probability and statistical probability
into a single judgment or number is essential. A simple equation will suffice to represent
this notion:
pTotal = pTheory *
pChance
If the roulette wheel is fair, then pTheory =1, and therefore
pTotal is dominated by calculating the odds.
If the fastest horse wins, then pChance is 1, and pTotal is dominated by
pTheory. When gamblers bet on a horse,
converting horse theory to a numerical value is exactly what they do. Poker players have a tougher calculation –
not only do they have to know the odds of a card turning up, they also have to
assign a probability to the notion that bluffing will work, or that their
opponent is bluffing.
Betting on the Series
But once the bet becomes about the long run, or about public
health instead of an individual, then the calculation is quite different. It’s a two dimensional problem where the
primary goal is to predict the frequency of a result or different results, and
there will also be uncertainty about estimated frequencies. The probability calculation isn’t the same
any more. The probability of chance is
often a statistical frequency instead.
In fact, it may or may not be a theoretical frequency. For example, there can be a range of
statistical estimates that range from purely empirical to purely
theoretical. An historical record with a
large number of observations may justify a frequency estimate with no
theoretical uncertainty. On the other
hand, a fewer number of observations may serve to support a statistical theory
instead, which begets theoretical uncertainty.
The frequency calculation is now a function instead of a single number, so
the relationship between theoretical probability and the frequency of
occurrence is now something far more complicated:
p(Frequency) =
pTheory(pChance)
Empirical observations may also be used to disprove a theory
too. For example, a large number of observations
may show a particular die to be unfair.
The again, there may only be enough data the favor one theory over
another without being able to conclusively decide that one is indubitably
correct. That means you are going
to need a probability tree.
Quantifying Theoretical Probabilities
Frequentist probability schemes tend to acknowledge theoretical
uncertainty (e.g. as “systematic error”), but then go on to ignore it. On the other hand, Bayesian probability
schemes typically treat theoretical and statistical probabilities interchangeably. If you are betting on a single instance, that
works reasonably well. Updating a
theoretical prior with data can gradually transform the probability into one of
chance – the more data there are, the less the theory matters. But it isn’t really very scientific. If they were used to discriminate among
alternative theories, the data might be put to better use. That problem is even more critical for the
estimation of long run frequencies.
Updating the parameter estimates for a model that has been proven to be
wrong doesn’t make much sense.
Since it really is more consistent with how scientific
knowledge is developed, explicitly assigning probabilities to theories is a better
strategy for long-term issues where knowledge may be expected to progress. Since theoretical probabilities are
inherently subjective, it is hard to improve upon convening a panel of experts
to weigh the scientific evidence. Even
if the experts don’t get it quite right, or they aren’t the right sort of
experts, the process of assigning probabilities to competing theories creates an occasion
for scientific discussion. As
long as no one thinks that probabilities assigned to theories are the gospel
truth, it’s all good in my book.
As a recent example,
Trasande et al (2015) provided an overview of the efforts to characterize the
theoretical probabilities for causal theories involving potential health
effects of Endocrine Disrupting Chemicals (EDCs):
We now describe the general methods used to attribute disease and disability to EDCs, to weigh the probability of causation based upon the available evidence, and to translate attributable disease burden into costs. During a 2-day workshop in April 2014, five expert panels identified conditions where the evidence is strongest for causation and developed ranges for fractions of disease burden that can be attributed to EDCs.
I have more than a few quibbles with exactly what they did,
ranging from how the problems were characterized in the first place (i.e. by
presuming independent attributable risks), the
use of implausible dose-response models, the
lack of serious consideration of other (i.e. non-EDC) causal factors, and
the relationship between association and causation is all-or-none. Also, because the probability assignments are
subjective, a two-day workshop of experts with similar interests is not really sufficient
for a decision involving the economic impacts that are alleged, so I don’t
recommend taking these estimate as the last word. However, praise for the process is well deserved. Nonetheless, as it pertains to the present
topic of discussion, there is one error in how the theoretical probability was
employed after it was arrived at that must not go unnoticed:
Finally, recognizing that attributable cost estimates were accompanied by a probability, we performed a series of Monte Carlo simulations to produce ranges of probable costs across all the exposure-outcome relationships, assuming independence of each probabilistic event. Separate random number generation events were used to assign 1) causation or not causation, and 2) cost given causation, using the base case estimate as well as the range of sensitivity analytic inputs produced by the expert panel. To illustrate with an example, for an exposure-outcome relationship with an 80% probability of causation, random values between 0 and 1 in each simulation led to the first step, which either assigned no costs (random value ≤ 0.2) and costs (random value > 0.2).
If the problem required the combination of both theoretical
and statistical probabilities, the use of the probability tree in a Monte-Carlo
simulation would be appropriate. However,
there is a problem in implementation that arises from the fact that a causal
probability is NOT a probability of chance: A theory is either true all the
time or false all the time, and the entire cost estimate is dependent (so, no
you can’t assume independence) on the truth of the theory. So, using a causal probability to calculate
the probability of an event is inappropriate.
Instead, the logic should go like this: Since all of the causal
probabilities have a probability of less than 95%, the lower bound cost
estimate of all of the end points should be zero (see table
four). For those endpoints with a
causal probability of less than 50%, the central estimate should be zero as
well.
Reference
Trasande L, Zoeller RT, Hass U, Kortenkamp A, Grandjean P, Myers
JP, DiGangi J, Bellanger M, Hauser R, Legler J, Skakkebaek NE, and Heindel JJ
(2015). Estimating Burden and Disease
Costs of Exposure to Endocrine-Disrupting Chemicals in the European Union. J Clin
Endocrinol Metab 100: 1245–1255.
Official Post Soundtrack
Cars, The (1978). All
Mixed Up. In: The Cars, Track 9.
Post Notes
Thesis Post #64. If someone can figure out a way to short their bet on all those IQ points, I'm all in.
No comments:
Post a Comment