2 ms·

Let's take two examples - Ohio (a contested state with a lot of polls) and Alabama (only one poll since 1st August, but not contested). In Ohio there were 44 p

Let's take two examples - Ohio (a contested state with a lot of polls) and Alabama (only one poll since 1st August, but not contested). In Ohio there were 44 polls in the month preceding the election, with a mean of 51.4% Obama, 48.6% Romney. If the confidence interval each poll is 4%, then the interval for a single poll is 4% / sqrt(44) = 0.6%, which is easily enough to make a confident forecast of an Obama victory in Ohio (my model was > 98% confident in its Ohio forecast). In Alabama, the last poll was on Aug 16th, and it was 40% Obama, 60% Romney. Even with a margin of error of 4%, this state was clearly going to go to Romney (my model had 99.9% confidence in this forecast). This pattern is repeated for almost every state - the states with few polls are not contested, and the hotly contested states are a focus for pollsters, so there are a lot of polls there. The only really difficult states were Florida, Colorado and North Carolina, where the candidates polled so similarly that even with 30-40 polls you didn't have sufficient sample size to make strong forecasts.

bryanlarsen 14y ago
It doesn't work like that. Polls are not independent. If poll A is out by a large amount, then it's much more likely that poll B is also out. As well, the confidence interval assumes that the sample was a fair sample, which is highly unlikely. It's very very difficult to get a true random sample of the voting populace. It's hard enough to get a random sample of the populace, and a random sample of those who vote is even harder. That's why Nate Silver said that Romney had about a 10% chance of winning. Statistically it was much lower, but there was a very real chance of systematic polling errors.
- crntaylor 14y ago
  Polls are not independent. If poll A is out by a large amount, then it's much more likely that poll B is also out. That's a bias issue - I was only addressing sampling variation. If you wnat your model to address bias as well, of course you can build that in, given some priors on what the distribution of the bias is likely to be. I didn't bother in my model (which is why I was forecasting 98-99% chance of an Obama victory). We could discuss potential poll bias as well, but I thought that was a bit too much for this short comment.

bryanlarsen 14y ago

It doesn't work like that. Polls are not independent. If poll A is out by a large amount, then it's much more likely that poll B is also out. As well, the confidence interval assumes that the sample was a fair sample, which is highly unlikely. It's very very difficult to get a true random sample of the voting populace. It's hard enough to get a random sample of the populace, and a random sample of those who vote is even harder. That's why Nate Silver said that Romney had about a 10% chance of winning. Statistically it was much lower, but there was a very real chance of systematic polling errors.

crntaylor 14y ago
Polls are not independent. If poll A is out by a large amount, then it's much more likely that poll B is also out. That's a bias issue - I was only addressing sampling variation. If you wnat your model to address bias as well, of course you can build that in, given some priors on what the distribution of the bias is likely to be. I didn't bother in my model (which is why I was forecasting 98-99% chance of an Obama victory). We could discuss potential poll bias as well, but I thought that was a bit too much for this short comment.