A Dictionary of Probability

Probability Theory

Pascal and Fermat did not invent games of chance in 1654.  Those had been around at least since Caesar crossed the Rubicon.  The underlying mathematical theory wasn’t entirely unknown either (Hacking, 1975).  However, Pascal did help Fermat to develop the mathematics that could be used to apportion the winnings for an unfinished game of chance.  But apart from his mathematical contributions, perhaps the most notable thing that Pascal did was to give a new branch of mathematics its name.  It could have called something else, like “Aleatory Mathematics”.  But, Pascal had a strong interest in juridical matters, and he wanted to counter the Jesuit doctrine of Probabilism with an argument that would have the force of logic.   So, he called it Probability Theory.  Considering the confusion that has ensued since, that was perhaps inadvisable.

Frequency Theory

Francis Galton didn’t invent statistics in 1889 either (Porter, 1986).  That event has been attributed to the publication of “Observation upon the Bills of Mortality” by John Gaunt in 1662.  The name of “statistics” came about in the late 18th and early 19th century as the practice producing of census tables describing the frequency of various and sundry population characteristics among the members of the state began in Europe.  Nor was he the first to discover the utility of using mathematical probability theory for characterizing a range of statistical values, that development happened in the British insurance industry about 50 years earlier.  Nor was he the first to use probability theory in science; the concept of measurement error had already been developed by that time. But Galton did introduce the concept of using probability theory as a scientific theory that describes frequency.  The psychological switch that accompanied that transformation was descried by Galton (1889) as follows:

It is difficult to understand why statisticians commonly limit their inquiries to Averages, and do not revel in more comprehensive views. Their souls seem as dull to the charm of variety as that of the native of one of our flat English counties, whose retrospect of Switzerland was that, if its mountains could be thrown into its lakes, two nuisances would be got rid of at once.

Although Galton put frequency theory to use in genetics, his idea didn’t take long to catch on in physics and other fields as well (Porter, 1986):

Although the most sophisticated statistical mathematics of the early nineteenth century grew out of the use of error theory in astronomy and related fields, the most fruitful applications of statistical reasoning proved to be elsewhere.  So long as probability functions were held to represent the imperfections of measurement and observation, there was little reason to do more with the variation that emerged than find ways to estimate it and eliminate as much of its effect as possible.  The source of modern statistics is to be found less in error analysis than in the use of probability as a modeling tool to capture and analyze real variation in nature and statistics.

The Words of Risk Analysis

Today, Probability and Statistics are introduced together as a math course, which gives the impression that probability, statistics, and the math are all pretty much the same thing (Hacking, 1975):

Philosophically minded students of probability nimbly skip among these different ideas, and take pains to say which probability concept they are employing at the moment.  The vast majority of practitioners of probability do no such thing. They go on talking of probability, doing their statistics and their decision theory oblivious to all this accumulated subtlety.

This is still true, but there is a significant minority of practitioners who do find it useful to distinguish concepts of probability.  For, example Kaplan (1997) gave a keynote address at a Society of Risk Analysis meeting that gave outlined three concepts of probability, and he gave a list of words that fall in each category:

1. Statistical: Random, Variability, “Aleatory Probability”, Objective probability, stochastic ontological, “in the world” probability, reliability, chance, and risk
2. Bayesian: Belief, Subjective probability, uncertainty, confidence, epistemic probability, forensic probability, plausibility, credibility, “Evidence Based” Probability
3. Mathematical Probability: Formal probability, axiomatic probability

That’s a good start, but for the purposes of this blog/document, it has one definition too few. So, let’s go with this set of definitions:

1. Abstract Probability:  Mathematical Probability, Bayes Theorem, Probability Trees
2. Statistics: Frequency, Variability
3. Chance: Random, Stochastic, “Aleatory Probability”
4. Inductive Probability: Subjective Probability, Epistemic Probability, Plausibility, “Evidence Based” Probability 

When compared to Kaplan’s dictionary, there are two main differences:

•  First, Bayes theorem has been moved into the abstract category along with mathematical probability.  In addition to representing belief or uncertainty, the theorem can just as easily be given a frequentist interpretation, and it is often used for both uncertainty and variability.  Another reason to put the theorem there is that even though many people think otherwise (e.g. Daston, 1989 and Kaplan,1997), a probability tree may be a better choice for formally representing Inductive Probability. So that has been added to the collection of abstract structures.  
• Second, the category of statistical probability has been broken into Frequency and Chance.  It is true that the underlying basis for a frequency statement and a statistically-based probability statement is essentially the same.  But, the frequency statement is about a statistical fact about either a population or series of events.  The probability of chance is about what will happen in a single instance. This distinction is important because some of the words that while some of the words that Kaplan uses like ‘uncertainty’, ‘confidence’, and ‘probability’ can refer to some combination of both chance and induction, they do not refer to frequency of occurrence. 

References

Daston, LJ (1988).  The Probability of Causes.  In: Classical Probability in the Enlightenment. Princeton University Press, pp.226-295.

Hacking, I (1975).  Duality.  In: The Emergence of Probability.  Cambridge University Press, pp. 11-17.

Kaplan, S (1997).  The Words of Risk Analysis.  Risk Anal 17:407-411.Porter, TM (1986).  The Rise of Statistical Thinking 1820-1900. Princeton University Press.

Official Sound Track

King Crimson (1981).  Elephant Talk.  In: Discipline, Track 1.

Post Note

This is thesis post #4.  Like #3, this follows #2 as an alternative bifurcation.