We don’t know anything. What do we do?
This article will require a basic intuition about probability, and to do the computational work, we’ll need some calculus. I will walk through the basic process, and the idea itself is intuitive.
There are times where we have absolutely no clue of what’s going to happen. What do we do if the sun doesn’t rise? What do we do if the moon drops into the ocean? What do we do if the world collapses? For that, we need to get a good sense of our next choice, and the odds that something like this happens. If we only have a sense of a sample size, how do we estimate the chance of something not happening?
For this, we turn to Laplace’s Rule of Succession:
Our assumptions:
1. Each event has the same chance, p, of happening
2. That chance is independent of whatever values the other events have
3. The ‘prior distribution’ over p is uniform (the same everywhere). f(p) = 1, where p is between 0 and 1
A quick detour into Bayes’s Rule:
Notice we use the term prior distribution. That is effectively what we think the possibilities of the data could be like before something happens. We’re choosing a simple prior distribution to not affect what our distribution will be like after that something happens.
All this stuff about prior and events happening is from Bayes’s Rule.
This rule is really useful for evidence:
- P(B) is known as the marginal, or the probability of the evidence under all possible hypotheses
- P(A) is the prior, or the probability that the hypothesis happens before given the evidence
- P(B|A) is the likelihood, or the probability that the evidence is true given the hypothesis
- P(A|B) is the posterior, or the probability that the hypothesis is true given the evidence
Now, let’s see what happens. We’ll need some basic rules of probability (all events add up to 1), and some rules of Calculus. Here’s the full version and where I got the intuition:
Step 1: Let us set up what we need to prove:
Notice how we need to show that the chance for us not getting an event so far is the probability that the event doesn’t happen over and over again.
We now need a way to prove the denominator. Once again, we start with the law of total probability.
Notice we use a substitution trick for p, allowing us to get a better value for our integral. We once again use Bayes’s rule to get the intuition behind how this works, where we look for the possible values of our events given that no event has happened so far.
Let’s substitute again:
What does this mean? Effectively, if you have no idea of the chances of an event happening, a good estimate is 1/(number of events + 2). For example, if a surgeon has had 98 successful heart transplants, a fair estimate could be that the chance of failure for the next surgery is 1/100, or 0.01.
Now, this works in theory, but it may not work in practice. Look at the assumptions; each event has to have the same chance of happening, which may not be true. The surgeon’s chance of success could be different if they didn’t have lunch. For assumption 2, the surgeon may have more confidence with more surgeries, which could mean the odds of failure are even lower than predicted. But here, our independence assumption fails. For further limitations, check this article out.
But in general, thanks to Laplace’s rule, we have a good intuition on what to do if we have no information other than sample size. Now, go guess!
