4 ms·

Not 100% sure, but for me "The three coins (non) example" is false. You could easily constract a random variable wich flips the free coins at random. Then you g

Not 100% sure, but for me "The three coins (non) example" is false. You could easily constract a random variable wich flips the free coins at random. Then you get again an iid sequence. I'm writing this basically because I think it is troubeling how "blogs" work. Explaining, lets say complex stuff, with simple diagrams. Especially in this case it even looks wrong and gives you the wrong intuition.

Schiphol 7y ago
The RV you suggest you could construct is not how the example works. From the article: >Let's consider the following scenario: say we have three coins with different biases (their probability of coming up heads): 0.4, 0.5 and 0.6. We pick one of the three coins at random, toss it 300 times and count the number of heads. So, not choosing a random coin every time, but choosing it once and tossing it 300 times.
- MichaelRazum 7y ago
  Maybe that is correct. What I'm just missing is a simple "proof" or at least an idea of a proof why some assumptions are violated. In this case you could a gain argue: picking a coin random and then toss 300 times --> Multidimensional CLT. Not saying the article is wrong. I'm just saying, where should be a proof or at least a bit more evidence... Just a simple plot is not enough from my perspective
  - Schiphol 7y ago
    The CLT holds for independent, identically distributed RVs. In the three-coin example the RVs are independent but not identically distributed: there are three distributions, one for each coin.
    - bonoboTP 7y ago
      No, it's the opposite: the 300 tosses are identically distributed, but they are not independent. They aren't independent because they all depend on a common cause: the choice of which coin to flip. Their property is also called "exchangeable" and it implies identical marginal distributions.
      - Schiphol 7y ago
        Right, thanks. The averages are independent and not identically distributed, but that's irrelevant to the theorem, right?
      - deleted 7y ago
        [deleted]
  - inimino 7y ago
    The violated assumption is simply i.i.d. of the so-called "random variable", in this case the individual coin flips would need to be i.i.d. for the CLT to hold. Obviously that's not the case at all by construction of the experiment. So really, it would be extraordinary if the plot showed anything other than it does. While the explanation of why it doesn't apply was handwavy ("can't really be written as a sum in the same way as before"), it is certainly correct and the graph helps give the intuition.
em500 7y ago
You're correct, and the blogpost (and several replies) are wrong. The sum or sample average of a simple mixture of Bernoulli distributions still converges to a Normal. There are too many errors and misconceptions in this blog and thread, confusion between mixture distributions and the sample mean of said mixture, confusion between sufficient and necessary conditions (IID is sufficient to proof the Lindeberg–Levy CLT, but it's not necessary for more general forms Lindeberg-Feller, Lyapunov). Unfortunately I'm at work and don't have time for a comprehensive rebuttal today. But people with some numpy or R skills can simply do the simulations themselves and see that the blog is not correct.
- LaserPineapple 7y ago
  > The sum or sample average of a simple mixture of Bernoulli distributions still converges to a Normal. The procedure given in the example is not a sum of samples from a mixture of Bernoulli distributions. It is a mixture of sums of Bernoulli distributions.
  - em500 7y ago
    The writing is not very clear, but I understand it to be a mixture of Bernoulli RVs: > Let's consider the following scenario: say we have three coins with different biases (their probability of coming up heads): 0.4, 0.5 and 0.6. We pick one of the three coins at random, toss it 300 times and count the number of heads. What is the distribution obtained? This is approximating the sum of 300 Bernoulli RVs with a Normal, which is perfectly valid. > As we have seen, if we fix the coin we're tossing, the number of heads can effectively be approximated by a distribution N(300p,300p(1−p)) (where p is the coin's bias). This time, however, each time we take a sample we might be tossing any of the three different coins. I understand the procedure here to be (1) choosing one of the 3 coins at random and tossing it, (2) repeating step (1) 300 times and summing the resulting number of heads. In this case the CLT does apply: the distribution of the sum-of-number-of-heads is approximately Normal, not the plotted tri-modal density.
    - LaserPineapple 7y ago

Schiphol 7y ago

The RV you suggest you could construct is not how the example works. From the article: >Let's consider the following scenario: say we have three coins with different biases (their probability of coming up heads): 0.4, 0.5 and 0.6. We pick one of the three coins at random, toss it 300 times and count the number of heads. So, not choosing a random coin every time, but choosing it once and tossing it 300 times.

MichaelRazum 7y ago
Maybe that is correct. What I'm just missing is a simple "proof" or at least an idea of a proof why some assumptions are violated. In this case you could a gain argue: picking a coin random and then toss 300 times --> Multidimensional CLT. Not saying the article is wrong. I'm just saying, where should be a proof or at least a bit more evidence... Just a simple plot is not enough from my perspective
- Schiphol 7y ago
  The CLT holds for independent, identically distributed RVs. In the three-coin example the RVs are independent but not identically distributed: there are three distributions, one for each coin.
  - bonoboTP 7y ago
    No, it's the opposite: the 300 tosses are identically distributed, but they are not independent. They aren't independent because they all depend on a common cause: the choice of which coin to flip. Their property is also called "exchangeable" and it implies identical marginal distributions.
    - Schiphol 7y ago
      Right, thanks. The averages are independent and not identically distributed, but that's irrelevant to the theorem, right?
    - deleted 7y ago
      [deleted]
- inimino 7y ago
  The violated assumption is simply i.i.d. of the so-called "random variable", in this case the individual coin flips would need to be i.i.d. for the CLT to hold. Obviously that's not the case at all by construction of the experiment. So really, it would be extraordinary if the plot showed anything other than it does. While the explanation of why it doesn't apply was handwavy ("can't really be written as a sum in the same way as before"), it is certainly correct and the graph helps give the intuition.

em500 7y ago

You're correct, and the blogpost (and several replies) are wrong. The sum or sample average of a simple mixture of Bernoulli distributions still converges to a Normal. There are too many errors and misconceptions in this blog and thread, confusion between mixture distributions and the sample mean of said mixture, confusion between sufficient and necessary conditions (IID is sufficient to proof the Lindeberg–Levy CLT, but it's not necessary for more general forms Lindeberg-Feller, Lyapunov). Unfortunately I'm at work and don't have time for a comprehensive rebuttal today. But people with some numpy or R skills can simply do the simulations themselves and see that the blog is not correct.

LaserPineapple 7y ago
> The sum or sample average of a simple mixture of Bernoulli distributions still converges to a Normal. The procedure given in the example is not a sum of samples from a mixture of Bernoulli distributions. It is a mixture of sums of Bernoulli distributions.
- em500 7y ago
  The writing is not very clear, but I understand it to be a mixture of Bernoulli RVs: > Let's consider the following scenario: say we have three coins with different biases (their probability of coming up heads): 0.4, 0.5 and 0.6. We pick one of the three coins at random, toss it 300 times and count the number of heads. What is the distribution obtained? This is approximating the sum of 300 Bernoulli RVs with a Normal, which is perfectly valid. > As we have seen, if we fix the coin we're tossing, the number of heads can effectively be approximated by a distribution N(300p,300p(1−p)) (where p is the coin's bias). This time, however, each time we take a sample we might be tossing any of the three different coins. I understand the procedure here to be (1) choosing one of the 3 coins at random and tossing it, (2) repeating step (1) 300 times and summing the resulting number of heads. In this case the CLT does apply: the distribution of the sum-of-number-of-heads is approximately Normal, not the plotted tri-modal density.
  - LaserPineapple 7y ago