I find Bayes' theorem is a bit more intuitive when written out not for probabilities but for the odds ratios, i.e. the ratio of the probabilities of something being true and something being false. Quite often one of these two will be close to zero, meaning the odds ratio will be close to (the inverse of) the probability anyway. So, letting H be the hypothesis and E the evidence, Bayes' theorem looks like
o(H|E) = o(H) P(E|H)/P(E|¬H)
Or in words, you update the odds ratio simply by multiplying it with the ratio of likelihoods of observing the evidence. Because odds ratios range through all positive numbers, you obviate the annoying normalization step you need with probabilities, which need to remain between 0 and 1.
Applied to the example in the article, a test with with a false positive ratio of 1% can only ever boost the odds ratio of your believe by a factor of 100 (and in the case of the article it's even only 98:1), so if a disease has a prevalence of about 2:10000, such a test can only bring it to about 2:100.
On the topic of finding Bayes' theorem intuitively, I can never remember it on its own, but starting from the joint probability P(A,B) for any arbitrary A and B always helps me:
// A and B are arbitrary.
P(A,B) = P(B,A)
// These two make sense if I sound them out in words.
P(A,B) = P(A) P(B|A)
P(B,A) = P(B) P(A|B)
// Combine to give Bayes' theorem.
P(A) P(B|A) = P(B) P(A|B)
The advantage of the odds formulation is that you can compute posterior odds in your head, whereas computing the posterior probabilities via the standard form of the theorem (that you wrote down here) involves an unpleasant normalization factor.
Exactly! The mathematical aspect of Bayes' Theorem is a simple algebraic manipulation of conditional probabilities.
Another way to intuitively justify your middle two equations is to visualise a Venn diagram with overlapping circles representing A and B, such that areas correspond to probabilities. Then P(B|A) is the area of the overlap - i.e. P(A,B) - divided by the area of A - i.e. P(A).
For me, taking the logarithm makes it even more intuitive somehow - the test becomes something like a modifier in an RPG-style game. In your example, the test gives +2 to the possibility of having a disease, a more powerful test could be +3, etc. Adding all the different +/- modifiers works just as well, and all this math is easy to do in one’s head.
Jaynes was a big proponent of taking the logarithm as well (as you probably know), referring to it as “measuring in decibels”. Unfortunately, I don’t actually understand what this log p/(1-p) gadget (“logit”?) actually does, mathematically speaking: it looks tantalizingly similar to an entropy of something, but I don’t think it is? Relatedly, I don’t really see how this would work for more than two outcomes—or why it shouldn’t.
The logit function is just a mapping from non-log prob space to log odds space. It’s the odds formula wrapped in a natural logarithm. In one way, it’s not “doing” much of anything. It’s not the journey, it’s the destination.
Why hang out in log odds space? Well, it’s a bigger space. It’s [-inf, inf], which is bigger than the [0, inf] of odds and the [0, 1] of prob. Odds space means you can multiply without normalizing. Log space means you multiply by adding. And being a natural log means you’re good to go when you start doing calculus and the like.
You can also cover a huge range of probabilities in a small range of numbers. -21 to 21 covers 1 in a billion to 1 - (1 in a billion).
which is admittedly not notation you can find in a high-school textbook, but you still understand what I mean.
One advantage is that it makes it abundantly clear that the restriction to two hypotheses (one of which is H, the other is then inevitably ¬H) is completely incidental and you absolutely can use the analogous expression for more (e.g. the three doors in the Monty Hall paradox). Another, funnier one is that it suggests that when talking about “odds” you’re using projective coordinates instead of the usual L1-normalized ones (and you are! your “o” is then the affine coordinate of course).
What I wanted to say is, the :-separated blocks are not fractions, they are two-component vectors up to a multiple (aka coordinates on [a part of] the projective real line, for those who like that sort of thing), and the “two” here is completely incidental.
For example, suppose that, in the Monty Hall paradox, you chose door one and observed Monty open door three. Then the posterior odds for the car being behind each door are calculated as
> the :-separated blocks are not fractions, they are two-component vectors up to a multiple
Projective coordinates are fractions, or more precisely the ℤ⁺²-projective vector space[1] (over the positive integers) is ℚ⁺, the positive rational numbers[0]. You generally want to use ℝ instead of ℤ for obvious reasons, but that's not really "not fractions" in any useful sense.
But writing it as vectors does have the benefit of making the generalization from ℝ² to ℝⁿ (n≥2) somewhere between obvious and trivial, whereas fractions are at-least-implicitly specific to the ℝ² case.
0: give or take some issues with 0 and ∞ that we don't care about because 0 and 1 are not probabilities
1: I'm sure someone can out-pedant me on the terminology here, but the point is it's two-dimensional, which (2) is the fewest dimensions this works non-degenerately for (leaving one degree of freedom, versus zero for the ℝ¹ case, and negative one (aka doesn't work at all) for ℝ⁰)
Applied to the example in the article, a test with with a false positive ratio of 1% can only ever boost the odds ratio of your believe by a factor of 100 (and in the case of the article it's even only 98:1), so if a disease has a prevalence of about 2:10000, such a test can only bring it to about 2:100.