I wish these types of fallacies, related to ignoring base rates and equating likelihoods with posteriors, were more widely appreciated.
However, a big problem in practice — and a reason why these fallacies exist aside from simple ignorance or mistake — is that the correct number, the posterior, requires knowing the prior base rates of something, which is often unknown. In some settings, the base rates are very well-characterized, but in others you really have no idea. In a lot of those cases, knowing the prior is qualitatively similar to knowing the posterior, which you're trying to figure out, so all you're left with with any certainty is the likelihood.
The fallacy exists, it's important, but sometimes I think there's a bit more to it than simple ignorance. Sometimes there's no information on prior rates, or you don't really know which prior distribution something comes from, there's a mixture for example.
I've never actually heard it called the "prosecutor's fallacy" before, but it should be obvious why. The sort of thing often being looked for is something like "is planning an insurrection against the government," "is a terrorist," "is a murderer." We don't know the true base rates for any of these things, but we do know the base rate is very low. Almost nobody is a terrorist or a murderer.
Also, it becomes easier to understand some types of frustrating or often wrong processes when we take into accounts not just rates of different error types, but the relative costs of them. A whole lot of criminals never see justice or get off on technicalities because the social cost in terms of destroying trust in the legal system is much higher for putting innocent people in prison than it is for failing to catch or failing to punish the guilty. Why do we seem to go so overboard with cancer screenings when the false positive rate is so high? Because the worst that can happen is mostly annoyance, wasted time, and wasted money. The worst that can happen with false negatives is you die. Why do our dragnets for terrorists seem to be so much more sensitive than dragnets for murder, rape, and property crime? Because even though the false positive rate here is even higher, the damage done by a successful terrorist attack is far greater. Why are FAANG hiring processes so jacked up? Because, right or wrong, the cost of hiring a bad engineer is perceived to be far higher than the cost of failing to hire a good one, especially when you get so many applicants that you're guaranteed to staff to the level you need virtually no matter how high a rate you reject at.
> the damage done by a successful terrorist attack is far greater.
Terrorist attacks are incredibly rare; murder, rape and (especially) property crime are commonplace, and don't rate even a column-inch in newspapers. Once upon a time, kids, there was a job called "court reporter").
How many people have you known who were victims of terrorist attacks? Right - zero. I don't know anyone who knows anyone who was the victim of a terrorist attack. How many people do you know who haven't been the victim of a personal crime, like assault, robbery or rape? Again, the expected answer is roughly zero.
Most successful applications of political violence (I hate the term "terrorism") result in just a handful of deaths/injuries. By "successful" I don't mean they achieved their objectives; I just mean the attack wasn't foiled before it happened.
> I've never actually heard it called the "prosecutor's fallacy" before, but it should be obvious why
The article goes into detail about this. If you look at the cases it links, some of them are pretty egregiously bad misuse of statistics to put people away. Sometimes while gaslighting the suspect at the same time.
I feel it could come down to not using statistics to infer a given conclusion.
For instance base probability accounted, if there was a 1 in a trillion chance someone was at the right place the right time, just straight assuming it couldn't happen by chance is still wrong. By definition that chance was not 0.
At some point a practical decision could be made to cut prosecution cost, but it should be understood that nothing was proven.
I think the legal system understands quite well that it doesn't prove anything (in the mathematical sense), which is why it has different standards of proof.
A 1 in 1 trillion chance would be considered both "beyond reasonable doubt" (enough for criminal matters) and satisfy the much weaker "balance of probabilities" usually applied to civil matters.
Of course there are plenty of examples to point to where people were convicted despite very reasonable doubt.
And if that even happens a lot in the world (e.g., many nurses looking at many patients every day), then that chance is quite likely to realize somewhere.
However, a big problem in practice — and a reason why these fallacies exist aside from simple ignorance or mistake — is that the correct number, the posterior, requires knowing the prior base rates of something, which is often unknown. In some settings, the base rates are very well-characterized, but in others you really have no idea. In a lot of those cases, knowing the prior is qualitatively similar to knowing the posterior, which you're trying to figure out, so all you're left with with any certainty is the likelihood.
The fallacy exists, it's important, but sometimes I think there's a bit more to it than simple ignorance. Sometimes there's no information on prior rates, or you don't really know which prior distribution something comes from, there's a mixture for example.