4 ms·

Ok, here's my null hypothesis: the election is not rigged. We are going to assume that this null hypothesis is true. Now, we run 30 tests on the data. What i

by jibiki 17y ago

Ok, here's my null hypothesis: the election is not rigged. We are going to assume that this null hypothesis is true. Now, we run 30 tests on the data. What is the probability that at least two tests reject the null hypothesis with 95% confidence? That's the question I'm answering.

earl 17y ago
I would say no: Because if the tests are independent, then something is well and truly broken! Your bernoulli CDF is for independent draws. Further, consider a hypothesis test. You feed it a desired chance of type I error -- say 0.05 -- and it gives you rejection regions for your test statistic. That is, it picks some subset of the domain of the test statistic and labels that as "reject H0" and labels the complement as "accept H0." Claiming that these 30 acceptance / rejection regions will somehow be independent of each other isn't true -- since our tests are accurate, or at least reasonably so, the accept / reject regions will be the same, or nearly so. Could it be possible to find tests with totally conflicting accept / reject regions? I'd say that's highly unlikely -- the regions would then be complements of each other. What is possible is that you will find paired tests with AR regions that are slightly different. But now you are asking the probability that your test statistic lands in conflicting areas of at least 2 pairs of these test regions. Again, this can be modeled -- try MCMC, perhaps -- but it will not be anything like bernoulli. Further, Does this make any sense?
- jibiki 17y ago
  > Claiming that these 30 acceptance / rejection regions will somehow be independent of each other isn't true -- since our tests are accurate, or at least reasonably so, the accept / reject regions will be the same, or nearly so. Ok, here's an example. Test 1 tests the following hypothesis: the last two digits of polling data are uniformly distributed amongst the values 00, 01, ..., 99. Test 2 tests the following hypothesis: the first digit is distributed according to Benford's law. For large data (like vote totals) these tests are (nearly) independent, no?

by jibiki 17y ago

earl 17y ago

I would say no: Because if the tests are independent, then something is well and truly broken! Your bernoulli CDF is for independent draws. Further, consider a hypothesis test. You feed it a desired chance of type I error -- say 0.05 -- and it gives you rejection regions for your test statistic. That is, it picks some subset of the domain of the test statistic and labels that as "reject H0" and labels the complement as "accept H0." Claiming that these 30 acceptance / rejection regions will somehow be independent of each other isn't true -- since our tests are accurate, or at least reasonably so, the accept / reject regions will be the same, or nearly so. Could it be possible to find tests with totally conflicting accept / reject regions? I'd say that's highly unlikely -- the regions would then be complements of each other. What is possible is that you will find paired tests with AR regions that are slightly different. But now you are asking the probability that your test statistic lands in conflicting areas of at least 2 pairs of these test regions. Again, this can be modeled -- try MCMC, perhaps -- but it will not be anything like bernoulli. Further, Does this make any sense?

jibiki 17y ago
> Claiming that these 30 acceptance / rejection regions will somehow be independent of each other isn't true -- since our tests are accurate, or at least reasonably so, the accept / reject regions will be the same, or nearly so. Ok, here's an example. Test 1 tests the following hypothesis: the last two digits of polling data are uniformly distributed amongst the values 00, 01, ..., 99. Test 2 tests the following hypothesis: the first digit is distributed according to Benford's law. For large data (like vote totals) these tests are (nearly) independent, no?