4 ms·

But in this case the samples are small. If you flip a coin once and got tails, what is the probability of heads?

by pps43 8y ago

joshuamorton 8y ago
That's not quite the same question as what I'm answering. I'm saying "given you flip a coin once and it comes up heads, what is the probability that its a fair coin" (or previously, given you flipped a coin 20 times, and it came up 15/5, what is the likelyhood it is a fair coin). And with a single flip, you can't actually answer that question (BetaDist[1, 0] is undefined). Now, once you've answered that question, you can integrate to get a probability that the next coin is heads, but with a single flip you can't even answer the first question. You need to have gotten heads and tails each at least once.
- isostatic 8y ago
  Flip a coin 50 times and get 50 heads, it's almost impossible it's a fair coin. Does your BetaDist function work in that case?
  - joshuamorton 8y ago
    No, the beta distribution BetaDist[a, b] is only defined for a, b strictly > 0. When a or b is very large and the other is zero, you can treat it as 1 and get a decent approximation, but it'll be a an estimate that biases towards the center.
  - joshuamorton 8y ago
    I should also mention that if you're willing to make assumptions like a uniform prior, this totally works, but it can lead to some surprising results for small a, b. Notably a uniform prior is Beta[1, 1]. Beta[2,1] implies something like a 75% chance of the true mean being above .5, which seems a tad overzealous.
- pps43 8y ago
  Exactly, that's not the question you're answering, but it is the question that is being asked in this context: what is the probability of rapid unplanned disassembly of SpaceX rockets.
  - joshuamorton 8y ago
    Hence the "but you can integrate to answer that". If your rocket has a 50% chance of being explosion proof and a 50% chance of blowing up half the time (you could imagine this as any number of situations, like two otherwise identical rockets which you use), any given launch has a 25% chance of rapid unplanned disassembly. You can do the same thing over a continuous random variable.
    - pps43 8y ago
      But you know neither the chance of the new rocket being explosion proof, nor the chance of the rocket blowing up half the time. You built the rocket, launched it once, and it did not blow up. What should be integrated to get the probability of rapid unplanned disassembly?
      - joshuamorton 8y ago
        Enter the Beta(variate) distribution. The beta distribution[1] is a cool statistical distribution defined by BetaDist{a,b} (or alpha, beta, but that's too much work), where a is the number of successes and b is the number of failures you've sampled. It has a number of cool properties, chief among them that given X = BetaDist{a,b}, then cdf(X, x) = the probability that the mean of the distribution you are approximating is less than x. It has a bunch of other nice properties too (like E[X] = a / (a + b), which should be obvious), but those aren't as relevant here. So let's say that you assume a uniform prior. This is defined as BetaDist{1,1} [2]. this is probably the wrong prior. So you might have a better idea. If, for example, you believe there is a 10% chance of your rocket exploding based on some calculations you've done, you might use a differently tuned beta distribution, like BetaDist{9,1} or BetaDist{4.5,.5} if you were feeling uncertain (but in general it would likely be better to use {8,2} in that situation iirc). But let's assume {1,1} for now. So you launch your rocket. Everything goes great. You update your distribution. Its a success. So you get BetaDist{2,1} [3]. So what is the chance your rocket explodes? Well the cdf of your beta distribution is the probability that the mean is less than x. So the pdf of the beta distribution is the probability that the mean is exactly x. So then The integral from 0 -> 1 of `(1 - x) * pdf(X, x) dx` is the estimated probability that your rocket explodes on its next launch, since that's "for every value x, the likelyhood of the distribution being that one multiplied by the chance your rocket explodes given that distribution". For the one rocket case, this happens to be equal to E[X] = a / (a + b), so it's 1/3. For the two rocket case, you apply reinforcement learning/k-armed bandit strategies like UBC1[4] or Thompson Sampling[5]. These are algorithms that will result in you picking the best rocket with as few unnecessary explosions as possible, provably. You can see some related discussion I've had on HN about these algorithms [6]. [1]: https://en.wikipedia.org/wiki/Beta_distribution https://en.wikipedia.org/wiki/Beta_distribution [2]: http://www.wolframalpha.com/input/?i=beta+distribution+(1,1) http://www.wolframalpha.com/input/?i=beta+distribution+(1,1) [3]: http://www.wolframalpha.com/input/?i=beta+distribution+(2,+1) http://www.wolframalpha.com/input/?i=beta+distribution+(2,+1... [4]: http://banditalgs.com/2016/09/18/the-upper-confidence-bound-algorithm/ http://banditalgs.com/2016/09/18/the-upper-confidence-bound-... [5]: https://en.wikipedia.org/wiki/Thompson_sampling https://en.wikipedia.org/wiki/Thompson_sampling [6]: https://news.ycombinator.com/item?id=17014232 https://news.ycombinator.com/item?id=17014232

But in this case the samples are small. If you flip a coin once and got tails, what is the probability of heads?

by pps43 8y ago

But in this case the samples are small. If you flip a coin once and got tails, what is the probability of heads?

That's not quite the same question as what I'm answering. I'm saying "given you flip a coin once and it comes up heads, what is the probability that its a fair coin" (or previously, given you flipped a coin 20 times, and it came up 15/5, what is the likelyhood it is a fair coin). And with a single flip, you can't actually answer that question (BetaDist[1, 0] is undefined). Now, once you've answered that question, you can integrate to get a probability that the next coin is heads, but with a single flip you can't even answer the first question. You need to have gotten heads and tails each at least once.

isostatic 8y ago
Flip a coin 50 times and get 50 heads, it's almost impossible it's a fair coin. Does your BetaDist function work in that case?
- joshuamorton 8y ago
  No, the beta distribution BetaDist[a, b] is only defined for a, b strictly > 0. When a or b is very large and the other is zero, you can treat it as 1 and get a decent approximation, but it'll be a an estimate that biases towards the center.
- joshuamorton 8y ago
  I should also mention that if you're willing to make assumptions like a uniform prior, this totally works, but it can lead to some surprising results for small a, b. Notably a uniform prior is Beta[1, 1]. Beta[2,1] implies something like a 75% chance of the true mean being above .5, which seems a tad overzealous.
pps43 8y ago
Exactly, that's not the question you're answering, but it is the question that is being asked in this context: what is the probability of rapid unplanned disassembly of SpaceX rockets.
- joshuamorton 8y ago
  Hence the "but you can integrate to answer that". If your rocket has a 50% chance of being explosion proof and a 50% chance of blowing up half the time (you could imagine this as any number of situations, like two otherwise identical rockets which you use), any given launch has a 25% chance of rapid unplanned disassembly. You can do the same thing over a continuous random variable.
  - pps43 8y ago
    But you know neither the chance of the new rocket being explosion proof, nor the chance of the rocket blowing up half the time. You built the rocket, launched it once, and it did not blow up. What should be integrated to get the probability of rapid unplanned disassembly?
    - joshuamorton 8y ago
      Enter the Beta(variate) distribution. The beta distribution[1] is a cool statistical distribution defined by BetaDist{a,b} (or alpha, beta, but that's too much work), where a is the number of successes and b is the number of failures you've sampled. It has a number of cool properties, chief among them that given X = BetaDist{a,b}, then cdf(X, x) = the probability that the mean of the distribution you are approximating is less than x. It has a bunch of other nice properties too (like E[X] = a / (a + b), which should be obvious), but those aren't as relevant here. So let's say that you assume a uniform prior. This is defined as BetaDist{1,1} [2]. this is probably the wrong prior. So you might have a better idea. If, for example, you believe there is a 10% chance of your rocket exploding based on some calculations you've done, you might use a differently tuned beta distribution, like BetaDist{9,1} or BetaDist{4.5,.5} if you were feeling uncertain (but in general it would likely be better to use {8,2} in that situation iirc). But let's assume {1,1} for now. So you launch your rocket. Everything goes great. You update your distribution. Its a success. So you get BetaDist{2,1} [3]. So what is the chance your rocket explodes? Well the cdf of your beta distribution is the probability that the mean is less than x. So the pdf of the beta distribution is the probability that the mean is exactly x. So then The integral from 0 -> 1 of `(1 - x) * pdf(X, x) dx` is the estimated probability that your rocket explodes on its next launch, since that's "for every value x, the likelyhood of the distribution being that one multiplied by the chance your rocket explodes given that distribution". For the one rocket case, this happens to be equal to E[X] = a / (a + b), so it's 1/3. For the two rocket case, you apply reinforcement learning/k-armed bandit strategies like UBC1[4] or Thompson Sampling[5]. These are algorithms that will result in you picking the best rocket with as few unnecessary explosions as possible, provably. You can see some related discussion I've had on HN about these algorithms [6]. [1]: https://en.wikipedia.org/wiki/Beta_distribution https://en.wikipedia.org/wiki/Beta_distribution [2]: http://www.wolframalpha.com/input/?i=beta+distribution+(1,1) http://www.wolframalpha.com/input/?i=beta+distribution+(1,1) [3]: http://www.wolframalpha.com/input/?i=beta+distribution+(2,+1) http://www.wolframalpha.com/input/?i=beta+distribution+(2,+1... [4]: http://banditalgs.com/2016/09/18/the-upper-confidence-bound-algorithm/ http://banditalgs.com/2016/09/18/the-upper-confidence-bound-... [5]: https://en.wikipedia.org/wiki/Thompson_sampling https://en.wikipedia.org/wiki/Thompson_sampling [6]: https://news.ycombinator.com/item?id=17014232 https://news.ycombinator.com/item?id=17014232