4 ms·

It's easier to compute the probability that less than 2 tests report a value of 5%. This is: (30 choose 0)(.95^30) + (30 choose 1)(.95^29)*(.05^1) = 0.

by jibiki 17y ago

It's easier to compute the probability that less than 2 tests report a value of 5%. This is: (30 choose 0)*(.95^30) + (30 choose 1)*(.95^29)*(.05^1) = 0.553542075 So there's about a 50-50 chance (obviously I'm assuming that the tests are independent of each other.) The relevant Wikipedia article: http://en.wikipedia.org/wiki/Binomial_distribution http://en.wikipedia.org/wiki/Binomial_distribution

itistoday 17y ago
"less than 2"? Assuming you're doing the math correctly, it means that there's a 50-50 chance that "0 or 1" of the tests "report a value of 5%" (and does that mean less than or equal to 5%?). Which, again, doesn't say very much... The question was: What is the chance of being able to find exactly 2 tests?
- jibiki 17y ago
  > The question was: What is the chance of being able to find exactly 2 tests Sure thing. That's: (30 choose 2) * (.95^28) * (.05^2) = 0.258636738 I thought you meant "at least two tests", for obvious reasons. (If 4 tests showed vote rigging, the researchers would still report vote rigging...) > (and does that mean less than or equal to 5%?) Yes.
  - itistoday 17y ago
    Thanks, looks like I'll have to brush up on my stats to analyze this further... But even without direct knowledge of how this works, shouldn't the "goodness" of the tests play a role in the numbers you're using? It doesn't seem to... > I thought you meant "at least two tests", for obvious reasons. Yes, sorry, I did, I guess you meant to say I should subtract 0.553542075 from 1 to get my answer. (just saw earl's post above...)
earl 17y ago
You are, frankly, speaking nonsense. As you pointed out, the tests will assuredly not be independent of each other. Second, what you have calculated is... I don't know. You are calculating the cumulative distribution function for a binomial distribution with n=30, p=0.95. How on earth does that relate to the previous post? Are you somehow confusing a p-value -- which is a statement about the minimal alpha for a rejection region such that given a set of data X and a test, we would reject our null hypothesis H_0 -- and a probability? Because they have almost nothing to do with each other.
- itistoday 17y ago
  Thanks earl for this post, I thought the calculation looked off... but I don't have the statistical background to say much. :-)
- jibiki 17y ago
  Ok, here's my null hypothesis: the election is not rigged. We are going to assume that this null hypothesis is true. Now, we run 30 tests on the data. What is the probability that at least two tests reject the null hypothesis with 95% confidence? That's the question I'm answering.
  - earl 17y ago
    I would say no: Because if the tests are independent, then something is well and truly broken! Your bernoulli CDF is for independent draws. Further, consider a hypothesis test. You feed it a desired chance of type I error -- say 0.05 -- and it gives you rejection regions for your test statistic. That is, it picks some subset of the domain of the test statistic and labels that as "reject H0" and labels the complement as "accept H0." Claiming that these 30 acceptance / rejection regions will somehow be independent of each other isn't true -- since our tests are accurate, or at least reasonably so, the accept / reject regions will be the same, or nearly so. Could it be possible to find tests with totally conflicting accept / reject regions? I'd say that's highly unlikely -- the regions would then be complements of each other. What is possible is that you will find paired tests with AR regions that are slightly different. But now you are asking the probability that your test statistic lands in conflicting areas of at least 2 pairs of these test regions. Again, this can be modeled -- try MCMC, perhaps -- but it will not be anything like bernoulli. Further, Does this make any sense?
    - jibiki 17y ago
      > Claiming that these 30 acceptance / rejection regions will somehow be independent of each other isn't true -- since our tests are accurate, or at least reasonably so, the accept / reject regions will be the same, or nearly so. Ok, here's an example. Test 1 tests the following hypothesis: the last two digits of polling data are uniformly distributed amongst the values 00, 01, ..., 99. Test 2 tests the following hypothesis: the first digit is distributed according to Benford's law. For large data (like vote totals) these tests are (nearly) independent, no?

It's easier to compute the probability that less than 2 tests report a value of 5%. This is: (30 choose 0)*(.95^30) + (30 choose 1)*(.95^29)*(.05^1) = 0.

by jibiki 17y ago

itistoday 17y ago

"less than 2"? Assuming you're doing the math correctly, it means that there's a 50-50 chance that "0 or 1" of the tests "report a value of 5%" (and does that mean less than or equal to 5%?). Which, again, doesn't say very much... The question was: What is the chance of being able to find exactly 2 tests?

jibiki 17y ago
> The question was: What is the chance of being able to find exactly 2 tests Sure thing. That's: (30 choose 2) * (.95^28) * (.05^2) = 0.258636738 I thought you meant "at least two tests", for obvious reasons. (If 4 tests showed vote rigging, the researchers would still report vote rigging...) > (and does that mean less than or equal to 5%?) Yes.
- itistoday 17y ago
  Thanks, looks like I'll have to brush up on my stats to analyze this further... But even without direct knowledge of how this works, shouldn't the "goodness" of the tests play a role in the numbers you're using? It doesn't seem to... > I thought you meant "at least two tests", for obvious reasons. Yes, sorry, I did, I guess you meant to say I should subtract 0.553542075 from 1 to get my answer. (just saw earl's post above...)

earl 17y ago

You are, frankly, speaking nonsense. As you pointed out, the tests will assuredly not be independent of each other. Second, what you have calculated is... I don't know. You are calculating the cumulative distribution function for a binomial distribution with n=30, p=0.95. How on earth does that relate to the previous post? Are you somehow confusing a p-value -- which is a statement about the minimal alpha for a rejection region such that given a set of data X and a test, we would reject our null hypothesis H_0 -- and a probability? Because they have almost nothing to do with each other.

itistoday 17y ago
Thanks earl for this post, I thought the calculation looked off... but I don't have the statistical background to say much. :-)
jibiki 17y ago
Ok, here's my null hypothesis: the election is not rigged. We are going to assume that this null hypothesis is true. Now, we run 30 tests on the data. What is the probability that at least two tests reject the null hypothesis with 95% confidence? That's the question I'm answering.
- earl 17y ago
  I would say no: Because if the tests are independent, then something is well and truly broken! Your bernoulli CDF is for independent draws. Further, consider a hypothesis test. You feed it a desired chance of type I error -- say 0.05 -- and it gives you rejection regions for your test statistic. That is, it picks some subset of the domain of the test statistic and labels that as "reject H0" and labels the complement as "accept H0." Claiming that these 30 acceptance / rejection regions will somehow be independent of each other isn't true -- since our tests are accurate, or at least reasonably so, the accept / reject regions will be the same, or nearly so. Could it be possible to find tests with totally conflicting accept / reject regions? I'd say that's highly unlikely -- the regions would then be complements of each other. What is possible is that you will find paired tests with AR regions that are slightly different. But now you are asking the probability that your test statistic lands in conflicting areas of at least 2 pairs of these test regions. Again, this can be modeled -- try MCMC, perhaps -- but it will not be anything like bernoulli. Further, Does this make any sense?
  - jibiki 17y ago
    > Claiming that these 30 acceptance / rejection regions will somehow be independent of each other isn't true -- since our tests are accurate, or at least reasonably so, the accept / reject regions will be the same, or nearly so. Ok, here's an example. Test 1 tests the following hypothesis: the last two digits of polling data are uniformly distributed amongst the values 00, 01, ..., 99. Test 2 tests the following hypothesis: the first digit is distributed according to Benford's law. For large data (like vote totals) these tests are (nearly) independent, no?