5 ms·

The frequentist answer is that you could get the proportion of tosses that come up heads arbitrarily close to 50% in a large enough sample. The Bayesian answer

The frequentist answer is that you could get the proportion of tosses that come up heads arbitrarily close to 50% in a large enough sample. The Bayesian answer is that given all the evidence available to me I would be willing to bet $1 to win $1 if the next toss comes up heads. (Since I work in finance I'll add that this assumes I'm risk neutral, so losing $1 is exactly as bad for me as winning $1 is good.) Except for the risk-neutrality detail this is all Probability 101, right? Or are you thinking of something else?

danbruc 10y ago
That the relative frequency converges to 50 % is obviously not true. You would have to be exceptionally lucky, but you could get heads, heads, tails repeated for ever and therefore the relative frequency of heads would fluctuate increasingly tiny amounts around 66.(6) %. This of course has probability zero, but it is not impossible. And there are of course many other sequences of outcomes for which the relative frequency does not converge to 50 %. So at best you could say that the relative frequency converges to 50 % with high probability, but now you have a circular definition because you make use of probabilities while defining probabilities. The Bayesian view is problematic for several reasons. Why do I need someone with believes about the coin, we are talking about intrinsic properties of tossing a coin after all. And if that is not enough, we also throw some betting in. Tossing a coin does certainly not depends on the invention of money and gambling, at least ignoring that coins are usually money. Last but not least you have to explain where your believe about a 50 % probability for heads comes from and how it is justified. I could certainly believe that the probability for heads is 25 %, that would not be a good believe.
- p1esk 10y ago
  So, what's the right answer?
  - danbruc 10y ago
    If I knew, I wouldn't have asked. And as far as I can tell nobody knows or at least there is no commonly accepted answer.
    - jgehrcke 10y ago
      I hope to have convinced with my previous answer that the mathematical definition of probability is key to resolving your paradox here. Mathematically (using the concept of infinity) and also physically (using the concept of large numbers) there is a simple and commonly accepted answer to your question. Since that definition is very basic, it is hard to find a direct answer to your question. You might find it interesting that in physics when we deal with large numbers that are not infinity it is usually good enough to know that a certain event is pretty damn unlikely (and it is essential to corroborate this with numbers as in "this happens once in 10^100 years"). In physics, we rarely have 100 % certainty. Logical consistency and a certainty that is large enough are usually key to success (i.e. growth of knowledge) in natural sciences.
      - prmph 10y ago
        I guess another way of stating the problem is this: say you toss a coin and get heads the first 1000 times. Now you are likely to believe that the coin is biased. But that cannot be proved. It may be that you have simply not tossed the coin enough times to perceive its fairness. Maybe after the 1000th toss you start to get enough tails such that after the millionth toss, it is not at all clear that there is any bias. So practically we can make a judgement as to whether the coin is biased on not based on how many tosses we think is sufficient, but theoretically it is impossible to distinguish a biased coin from a fair one if we toss to infinity
        danbruc 10y ago
        Yes, that is exactly what I had in mind. Probabilities are weired in the way that they say what will happen but then still leave open the possibility that this will not happen at all. You toss a coin a billion times, you will get about 500 million heads. Well, or you don't and there are exactly zero heads.
- hellogoodbyeeee 10y ago
  >That the relative frequency converges to 50 % is obviously not true. You would have to be exceptionally lucky, but you could get heads, heads, tails repeated for ever This would violate the law of large numbers. You may end up with the sequence HHT 10million times, but the chances that you continue to get that sequence for another 10million times is all but zero, and then gets even smaller as you add another 10million trials. As the number of trials approaches infinity, you will arrive at 50%.
  - danbruc 10y ago
    No, the law of large numbers does not assert that all sequences of outcomes converge, only that this happens almost surely. Or look at it the other way round, what mechanism would prevent heads, heads, tails repeated forever? I can certainly get heads, heads, tails on the first three tosses. After that I start over, three more tosses all independent of what just happened, again a 12.5 % chance for heads, heads, tails. Why could this not continue forever?
    - hellogoodbyeeee 10y ago
      I'm pretty sure your confusion about probability stems from you not understanding the mathematical concept of a "limit". Your sequence HHT has a 12.5% chance of happening, but so does the sequence TTH and as the number of trials approaches infinity, you will get similar counts of the two because they have equivalent probabilities.
      - danbruc 10y ago
        No, I will get similar counts with high probability but not surely. There are sequences of probability zero that do not converge. Think about it this way. Every coin toss in a sequence, finite or infinite, on its own can surly turn out to be heads, can't it? And all tosses are independent, aren't they? So why can't all tosses turn out heads? Unless you have a convincing argument why some tosses have to yield tails eventually, you have to deal with the fact that not all sequences of tosses converge to the expected probability.
- yellowstuff 10y ago
  I agree, there are problems and non-intuitive aspects to both approaches to probability. Otherwise there wouldn't be two approaches. I think your objection to frequentism unfairly conflates the idealized world with the physical world. Only in the idealized world can you assert a priori that a coin has 50% probability of coming up heads. In that world you can toss the coin infinite times and it almost surely comes up heads close to 50% of the time. In the physical world you merely have a model stating that heads will come up 50% of the time. In this world you could toss the coin millions of times and have it come up heads 66% of the time, but all you've done is provide really strong evidence that your model is wrong. Also, you left out 2 arguments against frequentism: it allows inconsistent beliefs, and in practice it has allowed bad approaches in scientific papers. As for the Bayseian view, being non-intuitive isn't the same as problematic. The Twins Paradox https://en.wikipedia.org/wiki/Twin_paradox https://en.wikipedia.org/wiki/Twin_paradox violates our intuitive understanding of how time works, but that's because our intuition is wrong in some conditions. You thought when I said the coin had 50% chance of coming up heads that I was making a statement about the coin, but really I was making a subjective statement about how I would bet. If you believe it's 25% then there's a clear way for us to resolve our different beliefs.
  - danbruc 10y ago
    I think the interesting part is not that the relative frequency might converge to something other than 50 % under non-ideal conditions, but that it might not converge at all, admittedly only very, very rarely. And this seems to force you into an infinite regress. 50 % probability for heads means that if you toss the coin infinitely often the relative frequency will converge to 50 %. But not quite, in very rare cases it won't. So you have to toss a coin infinitely often an infinite number of times and then you will see that all but a tiny fraction of the experiments indeed show the relative frequency converge to 50 %. But now you have to quantify that this tiny fraction is something of probability zero. And even worse, it is still possible that none of your repeated experiments showed convergence, you seem right back where you started. I would love to know to what you are referring with the inconsistent beliefs. I would not say that Bayesian view is non-intuitive, I would say it fails to account for important things. A priori probabilities have to be rooted somewhere. Where does you beliefe in a 50 % probability for heads come from? Because you have previously observed the relative frequencies of coin tosses? Because you made some theoretical observations about symmetries? There must be, at least so it seems to me, something about the probabilities associated with coin tosses that is independent of any individual, otherwise it would become rather difficult to explain how different individuals would arrive at similar probabilities independent of each other. So banning probabilities into the realm of beliefs does not cut it in my opinion.

danbruc 10y ago

That the relative frequency converges to 50 % is obviously not true. You would have to be exceptionally lucky, but you could get heads, heads, tails repeated for ever and therefore the relative frequency of heads would fluctuate increasingly tiny amounts around 66.(6) %. This of course has probability zero, but it is not impossible. And there are of course many other sequences of outcomes for which the relative frequency does not converge to 50 %. So at best you could say that the relative frequency converges to 50 % with high probability, but now you have a circular definition because you make use of probabilities while defining probabilities. The Bayesian view is problematic for several reasons. Why do I need someone with believes about the coin, we are talking about intrinsic properties of tossing a coin after all. And if that is not enough, we also throw some betting in. Tossing a coin does certainly not depends on the invention of money and gambling, at least ignoring that coins are usually money. Last but not least you have to explain where your believe about a 50 % probability for heads comes from and how it is justified. I could certainly believe that the probability for heads is 25 %, that would not be a good believe.

p1esk 10y ago
So, what's the right answer?
- danbruc 10y ago
  If I knew, I wouldn't have asked. And as far as I can tell nobody knows or at least there is no commonly accepted answer.
  - jgehrcke 10y ago
    I hope to have convinced with my previous answer that the mathematical definition of probability is key to resolving your paradox here. Mathematically (using the concept of infinity) and also physically (using the concept of large numbers) there is a simple and commonly accepted answer to your question. Since that definition is very basic, it is hard to find a direct answer to your question. You might find it interesting that in physics when we deal with large numbers that are not infinity it is usually good enough to know that a certain event is pretty damn unlikely (and it is essential to corroborate this with numbers as in "this happens once in 10^100 years"). In physics, we rarely have 100 % certainty. Logical consistency and a certainty that is large enough are usually key to success (i.e. growth of knowledge) in natural sciences.
    - prmph 10y ago
      I guess another way of stating the problem is this: say you toss a coin and get heads the first 1000 times. Now you are likely to believe that the coin is biased. But that cannot be proved. It may be that you have simply not tossed the coin enough times to perceive its fairness. Maybe after the 1000th toss you start to get enough tails such that after the millionth toss, it is not at all clear that there is any bias. So practically we can make a judgement as to whether the coin is biased on not based on how many tosses we think is sufficient, but theoretically it is impossible to distinguish a biased coin from a fair one if we toss to infinity
      - danbruc 10y ago
        Yes, that is exactly what I had in mind. Probabilities are weired in the way that they say what will happen but then still leave open the possibility that this will not happen at all. You toss a coin a billion times, you will get about 500 million heads. Well, or you don't and there are exactly zero heads.
hellogoodbyeeee 10y ago
>That the relative frequency converges to 50 % is obviously not true. You would have to be exceptionally lucky, but you could get heads, heads, tails repeated for ever This would violate the law of large numbers. You may end up with the sequence HHT 10million times, but the chances that you continue to get that sequence for another 10million times is all but zero, and then gets even smaller as you add another 10million trials. As the number of trials approaches infinity, you will arrive at 50%.
- danbruc 10y ago
  No, the law of large numbers does not assert that all sequences of outcomes converge, only that this happens almost surely. Or look at it the other way round, what mechanism would prevent heads, heads, tails repeated forever? I can certainly get heads, heads, tails on the first three tosses. After that I start over, three more tosses all independent of what just happened, again a 12.5 % chance for heads, heads, tails. Why could this not continue forever?
  - hellogoodbyeeee 10y ago
    I'm pretty sure your confusion about probability stems from you not understanding the mathematical concept of a "limit". Your sequence HHT has a 12.5% chance of happening, but so does the sequence TTH and as the number of trials approaches infinity, you will get similar counts of the two because they have equivalent probabilities.
    - danbruc 10y ago
      No, I will get similar counts with high probability but not surely. There are sequences of probability zero that do not converge. Think about it this way. Every coin toss in a sequence, finite or infinite, on its own can surly turn out to be heads, can't it? And all tosses are independent, aren't they? So why can't all tosses turn out heads? Unless you have a convincing argument why some tosses have to yield tails eventually, you have to deal with the fact that not all sequences of tosses converge to the expected probability.
yellowstuff 10y ago
I agree, there are problems and non-intuitive aspects to both approaches to probability. Otherwise there wouldn't be two approaches. I think your objection to frequentism unfairly conflates the idealized world with the physical world. Only in the idealized world can you assert a priori that a coin has 50% probability of coming up heads. In that world you can toss the coin infinite times and it almost surely comes up heads close to 50% of the time. In the physical world you merely have a model stating that heads will come up 50% of the time. In this world you could toss the coin millions of times and have it come up heads 66% of the time, but all you've done is provide really strong evidence that your model is wrong. Also, you left out 2 arguments against frequentism: it allows inconsistent beliefs, and in practice it has allowed bad approaches in scientific papers. As for the Bayseian view, being non-intuitive isn't the same as problematic. The Twins Paradox https://en.wikipedia.org/wiki/Twin_paradox https://en.wikipedia.org/wiki/Twin_paradox violates our intuitive understanding of how time works, but that's because our intuition is wrong in some conditions. You thought when I said the coin had 50% chance of coming up heads that I was making a statement about the coin, but really I was making a subjective statement about how I would bet. If you believe it's 25% then there's a clear way for us to resolve our different beliefs.
- danbruc 10y ago
  I think the interesting part is not that the relative frequency might converge to something other than 50 % under non-ideal conditions, but that it might not converge at all, admittedly only very, very rarely. And this seems to force you into an infinite regress. 50 % probability for heads means that if you toss the coin infinitely often the relative frequency will converge to 50 %. But not quite, in very rare cases it won't. So you have to toss a coin infinitely often an infinite number of times and then you will see that all but a tiny fraction of the experiments indeed show the relative frequency converge to 50 %. But now you have to quantify that this tiny fraction is something of probability zero. And even worse, it is still possible that none of your repeated experiments showed convergence, you seem right back where you started. I would love to know to what you are referring with the inconsistent beliefs. I would not say that Bayesian view is non-intuitive, I would say it fails to account for important things. A priori probabilities have to be rooted somewhere. Where does you beliefe in a 50 % probability for heads come from? Because you have previously observed the relative frequencies of coin tosses? Because you made some theoretical observations about symmetries? There must be, at least so it seems to me, something about the probabilities associated with coin tosses that is independent of any individual, otherwise it would become rather difficult to explain how different individuals would arrive at similar probabilities independent of each other. So banning probabilities into the realm of beliefs does not cut it in my opinion.