Reason consistently and you are already using probability.
Suppose you want to attach a number to how plausible a claim is, given what you know. You ask for only three things: plausibilities are real numbers, they move the way common sense says they should, and two valid ways of working out the same plausibility give the same answer.
Richard Cox showed in 1946 that these requirements leave no freedom. Up to a relabeling of the scale, your numbers must combine by the product rule and the sum rule, which are the rules of probability. E. T. Jaynes built his account of probability as extended logic on this result. Four figures come first, to build intuition. The proof follows.
Figure 1 · Two routes must agree
A rule for "and" has to give the same answer whichever way you group
You need a rule that combines two plausibilities into the plausibility that both claims hold. Call it F. Now take three claims in a chain: x is the plausibility of the first, y of the second given the first, and z of the third given the first two. To get the plausibility that all three hold you can combine the first two and then bring in the third, or combine the last two and then bring in the first. Both describe the same thing, so they must agree.
Each box is one use of the rule. The two top boxes are the two routes to "all three hold."
Averaging sounds reasonable but contradicts itself. Multiplying passes. So does the odd-looking rule. The weakest-link rule also passes this test, but it fails common sense in another way: make the stronger claim more plausible and the result does not move at all.
Figure 2 · Why a consistent rule has to be multiplication
A chain of equally plausible links makes a ladder
Figure 1 showed that grouping cannot matter. Here is what follows from that. Picture a message passed along a line of relays. Each relay, given that the message reached it intact, passes it on intact with the same plausibility u, a little short of certain.
How plausible is it that the message survives two relays? This is an "and" of two claims, each with plausibility u, so the rule gives F(u, u): the rule applied with u in both slots. For three relays, apply the rule to that result and u once more. Because it does not matter which neighboring pair you combine first, the plausibility of surviving k relays is one definite number. Call it rung k. Every extra relay lowers the plausibility, so the rungs form a ladder leading down from certainty, which is rung 0.
Now put a chain of m relays in front of a chain of n relays. Surviving both is an "and" of two claims with plausibilities rung m and rung n. But the two chains together are just one chain of m + n relays. So the rule applied to rung m and rung n has to give rung m + n. On the ladder, the rule for "and" does nothing except add rung numbers. Nothing was assumed about the rule beyond Figure 1, so this is true of every consistent rule.
rung m
rung n
the rule applied to them
rung m + n
Rung k is the plausibility that the message survives k relays. Top line: where the rungs fall on the plausibility scale, found by applying the rule over and over. Bottom line: the same rungs by number. The rungs are unevenly spaced in plausibility, and differently for each rule, but in rung numbers the rule always just adds.
A ladder only reaches some plausibilities. Move u toward 1 and the rungs crowd together, so a fine enough ladder comes as close as you like to any plausibility. In the limit every plausibility has a position on the ladder, and the rule for "and" adds positions. Making that limit precise is the technical part of the proof.
One last step turns adding into multiplying. Give rung 1 a label w between 0 and 1, and give rung k the label wk. Adding rung numbers is then the same as multiplying labels, as the last row of the table shows. For the multiply rule the labels are just the plausibilities themselves. For the odd-looking rule they are different numbers, and they still multiply. With these labels every consistent rule reads w(A and B) = w(A given B) × w(B). That is the product rule.
Figure 3 · A rule for "not"
The rule for "not" must undo itself, and must agree with the product rule
You also need a rule S that turns the plausibility of a claim into the plausibility of its denial. Two checks constrain it. Denying a denial returns the original claim, so applying S twice must give back what you started with. On the chart that means the curve is its own mirror image in the diagonal. And the product rule offers two ways to reach the plausibility of "A and B," one starting from A and one from B, which must agree.
the rule S- - its mirror image in the diagonal
Here x and y are the plausibilities of A and B for a pair of claims chosen so that both routes can be written with S alone. The proof gives the details.
Two of the four survive both checks. They look different, but Figure 4 shows they are the same rule on different scales.
Figure 4 · One more relabeling
Every surviving "not" rule becomes a straight line
The rules that pass both checks in Figure 3 form one family, w(A)m + w(not A)m = 1, with one curve for each positive m. Relabel the scale once more, as p = wm, and every curve in the family turns into the same straight line.
Left: the rule on the w scale, which bends as m changes. Right: the same rule after relabeling, which does not move. The dot is the same claim on both scales.
The relabeling does not disturb the product rule, because a power of a product is the product of the powers. So on the p scale both rules hold at once: p(A and B) = p(A given B) × p(B), and p(A) + p(not A) = 1. Those are the rules of probability.
The proof
Cox's theorem in seven steps
Write for the plausibility of given , for " and ," for " or ," and for "not ."
What is assumed
Numbers. Every is a real number, and a larger number means more plausible.
A rule for "and.", where is continuous and strictly increasing in each argument. To judge that and both hold, judge , then judge given .
A rule for "not.", where is continuous and strictly decreasing.
Consistency. Logically equivalent claims have the same plausibility, however it is worked out.
1. Grouping gives an equation for F
The claim can be read as or as . Put , , and apply the rule each way:
Consistency requires
2. Its solutions are products on a relabeled scale
The continuous, strictly increasing solutions of that equation are exactly
for some continuous, strictly monotonic, positive function . It is easy to check that any such satisfies the equation, since both sides equal . That there are no other solutions is the one fact quoted here without proof. Cox proved it assuming is differentiable. Aczél proved it without that assumption, in a much longer argument. Applying to both sides of the rule for "and" gives the product rule:
3. Certainty is 1 and impossibility is 0
If is certain given , then and are the same claim given , and is still certain given . The product rule becomes , so . If is impossible given , so is , and the product rule becomes for every . So is or . Take . The other choice is the same theory written with .
4. Consistency gives an equation for S
Since , we have . Now use the product rule and then the rule for "not":
The same steps with and exchanged give a second expression for the same quantity:
These must agree for all claims. Choose so that for some claim . Then and . Writing and , so that and , the two expressions become
5. Its solutions are one family
The solutions of that equation with are
Checking is one line: the left side becomes , which is unchanged when and are exchanged. That there are no other solutions is again quoted. Cox proved it assuming is twice differentiable, and Aczél proved it without that assumption. So
6. Relabel once more
Define . Raising the product rule to the power leaves its form unchanged, so
with and . Since is an increasing function of the original plausibility, it carries exactly the same ordering of claims.
7. The general sum rule follows
" or " is the denial of " and ." Apply the two rules in turn:
When and cannot both hold, the last term is zero and plausibilities add. Writing the product rule in both orders and dividing gives Bayes' theorem:
Fine print
What the theorem assumes and what it leaves open
One real number per claim. Approaches that use intervals or pairs of numbers fall outside the theorem.
The shape of the rule for "and." It depends only on the plausibility of one claim and of the other given the first. Jaynes reports that Myron Tribus checked the eleven possible forms against simple extreme cases, and that only this form and its mirror image, with the two claims exchanged, survive.
Enough smoothness, and enough plausibility values. Steps 2 and 5 need the functions to be well behaved and the plausibilities to fill out a range. Joseph Halpern showed in 1999 that the theorem can fail for a finite set of claims without such a condition. Later versions state it explicitly.
Additivity for finitely many claims only. The theorem does not give the rule for infinite sequences of claims.
No numbers to start from. The theorem says how plausibilities must combine. It does not say what any particular plausibility should be.
Sources:
R. T. Cox, "Probability, frequency and reasonable expectation," American Journal of Physics 14 (1946), pp. 1–13.
E. T. Jaynes, Probability Theory: The Logic of Science (2003), chapter 2, "The quantitative rules," which this page follows.
J. Aczél, Lectures on Functional Equations and Their Applications (1966), for the two functional equations without differentiability.
J. Y. Halpern, "A counterexample to theorems of Cox and Fine," Journal of Artificial Intelligence Research 10 (1999), pp. 67–85.