7 ms·

Just thought I'd share a tidbit that I haven't been able to elsewhere. I work for a large market research firm that does polls for a large news network. On Sund

Just thought I'd share a tidbit that I haven't been able to elsewhere. I work for a large market research firm that does polls for a large news network. On Sunday, we ran a poll for them that put Trump at 8 points ahead. They decided against publishing it.

pcunite 10y ago
Thank you for sharing. From my two months of non-professional research, it looked like Trump was going to win by a big margin. The major media outlets were looking outright deceptive from my perspective.
- mrlatinos 10y ago
  The irony is that the same network is now questioning how all the pollsters got it wrong... Frustrating.
  - SiVal 10y ago
    Interesting, isn't it? Filtering out facts that they didn't believe, not because they were trying to lie, but because those who demand that we "celebrate diversity" have so little actual diversity themselves that they all truly believed the same, standard narratives they consistently tell each other, resulted in a situation where one of their beliefs could be objectively tested, and they were all wildly wrong in exactly the same way. What does this suggest about all the other stories the mainstream media (CBS exec: "How could Nixon have won? I don't know a single person who voted for him!") tells every day, in exactly the same way, about all sorts of issues that can't be so easily measured? And what does it suggest about the academic institutions that enforce conformity to exactly the same set of opinions and narratives and not only filters information but fires and otherwise silences dissenters? And anyone who suggests that a system so utterly lacking in diversity of thought that there is no one left on campus who will tell them if they're wrong can't be trusted is called "anti-intellectual" and "anti-science". It doesn't tell me that they are wrong about everything, just that they are likely to be just as wrong about some other issues they are adamant about as they were about this one, but that promoting their dogma, not working to find the truth, is what they really care about, which doesn't make them wrong about any particular thing, just not reliable.
- sshumaker 10y ago
  pcunite - we've discussed in another thread, but the election was actually quite close. HRC had more of the popular vote, and a 1% shift from Trump to Clinton would have resulted in her victory. Here's an excellent analysis from five-thirty-eight: http://fivethirtyeight.com/features/what-a-difference-2-percentage-points-makes/ http://fivethirtyeight.com/features/what-a-difference-2-perc...
  - pcunite 10y ago
    I understand what is being reported now. I'm referring to my sense of awareness before the election. It looked like a big win - to me at least.
  - fixxer 10y ago
    Sure, but the polls being published and "marketed" had her with an insurmountable lead. Silver had her at 70% and that was generous to Trump compared to many. This isn't about the polls being wrong because the race was tight. Many misrepresented likely turnout (because many are incompetent and thought the 2016 electorate was going to be the 2012 electorate) and the media pumped up those same HRC favorable polls while the pundits attacked the validity of what turned out to be better polls (LA Times). It sure looked like a propaganda machine to me, but I'm just one of those deplorables.
    - grzm 10y ago
      Some honest questions: Do you think it's worth polling at all? Do you think it's possible they made honest mistakes in their models? If you're voting in the election, you're going to have an opinion. Is it possible to work on polling honestly while being an eligible voter? Personally, I think polling is worthwhile. I think it's possible they made honest (and sometimes incompetent, but not malicious) mistakes, and acknowledge that attempting to predict the outcome of a future event is inherently difficult. I think it is possible to do honest work while still voting.
      - fixxer 10y ago
        Polls are useful for the campaign as part of media optimization and field strategy. In terms of value to the electorate, I think polls are anti-value: they are presented to generate response, be it entertainment or motivation. The electorate needs to held accountable for turnout, not tricked into it. I don't believe in requiring voting, but I despise the methods used to GOTV. Either you possess civic engagement or you don't
- mikeash 10y ago
  It's amusing to see all the people coming out of the woodwork to claim that they knew all along that Trump would win big. The vote was almost dead even. If you though Trump was going to win big then you were just as wrong as the places that predicted a big Clinton win.
  - hga 10y ago
    Quibble: since this is done by Electoral College adjusted voting, it wasn't "almost dead even" by that accounting, as of right now the AP called Arizona for Trump, making it 290-228. Michigan leans him, New Hampshire leans Hillary, if those two break in those ways, it'll be 316-232, which is fairly big as these things go, close to Obama's 2012 win. About as big as I expected, although not quite as big as I hoped, then again, there are always surprises in Presidential elections. I didn't expect Trump to get Pennsylvania (or Toomey to win), I expected Russ Feingold to win (yesterday saw a report on the general issue of Team Hillary ignoring/taking for granted the Rust Belt, including ignoring a plea for help from Feingold), and one of the biggest surprises is that Minnesota truly was in play (margin looks to have gone from ~200K in 2012 to ~40K).
    - mikeash 10y ago
      The electoral college margin doesn't mean a whole lot. Tiny swings in the vote can result in huge swings in the electoral college. If you lose your balance on top of a mountain, teeter both ways, then fall to the right, the fact that you might come to rest miles from the peak doesn't mean it wasn't a near thing.
      - NhanH 10y ago
        If Clinton has lost only one of her firewall states, your comment would have applied. The fact that Trump flipped quite a few of them makes it no longer a close one. If two countries play a best-of-5 soccer games, and one beat the other 3-0 with three consecutive 1-0 games in the last minute. It would have been hard to claim that one team isn't obviously better and the margin of winning is small.
- PhantomGremlin 10y ago
  The major media outlets were looking outright deceptive from my perspective. I had some CNBC shows saved on my Tivo and I recently went back to look at what they were saying. Here's an extreme example of the deception. Just the day before the election CNBC was showing states such as Montana, North Dakota and South Dakota as "leaning" GOP, not "likely" GOP. But here are the results for those states: Montana Trump 57% Clinton 36% North Dakota Trump 64% Clinton 28% South Dakota Trump 61% Clinton 32% From this one could easily conclude that the mainstream media deliberately chose to deceive about the expected outcome in those states. Why? Perhaps to not show as much dark red on the national map? Yes there were true battlegrounds or toss-ups. E.g. Florida only went for Trump by about 1.3%. But there were more than 2x as many Trump voters than Clinton voters in North Dakota. It was a bald-faced brazen lie to call that "leaning".
forkandwait 10y ago
If I wanted to study a book for 10 hours on political polling and the math it takes to model the results, what book would you suggest?
- mrlatinos 10y ago
  I'm probably not the right person in the company to ask.
- dlss 10y ago
  The two best resources I'm aware of in this area are: * For designing psychology experiments: https://smile.amazon.com/Conducting-Research-Psychology-Measuring-Weight/dp/0495598194 https://smile.amazon.com/Conducting-Research-Psychology-Meas... * For Bayesian modeling and inference: https://smile.amazon.com/Bayesian-Analysis-Chapman-Statistical-Science/dp/1439840954 https://smile.amazon.com/Bayesian-Analysis-Chapman-Statistic... However, unless you are already very knowledgeable in these subjects, it will take longer than 10 hours to understand them. Very much worth the read though.
  - trendia 10y ago
    Depending on his / her statistics knowledge, Gelman could easily take 40 - 100+ hours to really understand.
  - forkandwait 10y ago
    Thanks, but that is a somewhat crazy answer. I have a BA in mat with a few upper division stat and prob classes, but desire to do a grad degree in stats (yuck). I just want to read a book about how pollsters practically execute their job.
    - sn9 10y ago
      You can usually find the methodology analysts like 538 use. The better ones are usually pretty transparent about them.
ubernostrum 10y ago
And with good reason: polls measure popular vote, not electoral vote, and Trump most certainly did not win the popular vote by 8 points. He didn't even come close to that margin.
- disposable099 10y ago
  Well, they shouldn't. The point of polls is to predict who's going to win. It's not that hard to take into account Electoral College anyways.
- MichaelBurge 10y ago
  I assume you're worried he didn't collect location data? Sometimes you can get the location data from their IP address or zip code(if it's a form).
  - ubernostrum 10y ago
    So if this poll said Trump was "8 points ahead", where did it say he was "8 points ahead"? A generic statement of "8 points ahead" is, when hearing about a poll with zero other context read to mean "8 points ahead in a national poll of likely voters with a representative sample", which is a national popular vote measure of limited utility in predicting electoral outcome.
    - mrlatinos 10y ago
      He was 8 points ahead in a national poll of likely voters with a representative sample when asked who they would be more likely to vote for. I don't have specifics besides the 8 pt. lead because I'm not on the analysis side of things and the results weren't published. And I agree, phone polling is fairly limited as far as outcome prediction goes. Part of the reason they didn't publish is because they thought it was a fluke.
      - ubernostrum 10y ago
        So, did he end up with an 8-percentage-point national popular-vote lead? If not, seems the poll was inaccurate.
youarefired 10y ago
That interesting. I run a silly little website website that invited people to vote against one of the two candidates [1]. I watched our analytics like a hawk during the couple of weeks before the elections in order to adjust messaging, content and ads. This, of course, wasn't scientific at all except for the fact that we were running ads to attract both sides of the discussion equally. They were geographically limited to the US. About a week before the elections the trend was unmistakable. The support for Trump in the form of CTR, engagement and votes to the hundreds of different ads out there told a story. We actually tried to get this story out and attempted to publish a press release. It was rejected by all of the top press release agencies we used for other business. Not one of them wanted to publish that things were not looking good from Clinton, even from a silly little entertainment site. To confirm the bias a few days later we issued a press release indicating that Clinton was catching-up and Trump was predicted to lose. Everyone published the release. We can't have the media operate with such bias in this country. Freedom of the press is important, but when the press become political activists we should all yell foul and reign them in. The first thing totalitarian regimes do is take control of the press. We can't have a press that is so far to the left that they won't report the truth, or worst, distort it. I know this is just one data point. There are plenty more from other sources. The bias and manipulation are very real. [1] http://youarefired.com/ http://youarefired.com/
- hga 10y ago
  We can't have the media operate with such bias in this country. Freedom of the press is important, but when the press become political activists we should all yell foul and reign them in. The first thing totalitarian regimes do is take control of the press. We can't have a press that is so far to the left that they won't report the truth, or worst, distort it. You say we "can't", but we demonstrably do have such a press. And you're 100% correct to discern the totalitarian nature of it. Which is why alternative sources have become so important, and as more people's "Pravda breakers" trip (https://news.ycombinator.com/item?id=12921328 https://news.ycombinator.com/item?id=12921328) the less important this old press becomes. Look no further than the former "Newspaper of Record", it's post-election front page, it's crashing revenue, and why I call it "Carlos Slim's personal blog". Historically it started with Reagan's zapping of the "'Fairness' Doctrine", which allowed right of center talk radio to become a thing, which I'm sure helped prime things for rise of the Internet. Drudge's 1998 reporting of a story spiked by Newsweek (sold in 2010 for a dollar and assumption of its liabilities) clearly marked this was a decisive change ... and yet our "press", MSM, this whole old establishment, seems determined to stay the course. Which, just citing the two financial bits above, looks like a path to oblivion (although many of us are predicting they'll get direct government support soon enough, but that won't get them readers), and therefore self-correcting in, say, the medium term. So I'm not quite so alarmed, as long as the Internet stays free of censorship at the IP and maybe DNS levels.
mikeash 10y ago
Given that your poll was off by 8 points, was that not the correct decision?
- freehunter 10y ago
  The correct way to present polls is with a sampling of many different polls, like Real Clear Politics or 538. Yes, all the polls were off across the board, but individual polls have a higher likelihood of being wrong than a bunch of polls combined.
- mrlatinos 10y ago
  Most of our other polls were off as well, but in Clinton's favor, so do we not publish those either? At the time, there was no way to know what was correct, so shouldn't sharing the information be more important than anything else?
  - mikeash 10y ago
    They were a lot closer, weren't they? 8 points off seems rather large.

pcunite 10y ago

Thank you for sharing. From my two months of non-professional research, it looked like Trump was going to win by a big margin. The major media outlets were looking outright deceptive from my perspective.

mrlatinos 10y ago
The irony is that the same network is now questioning how all the pollsters got it wrong... Frustrating.
- SiVal 10y ago
  Interesting, isn't it? Filtering out facts that they didn't believe, not because they were trying to lie, but because those who demand that we "celebrate diversity" have so little actual diversity themselves that they all truly believed the same, standard narratives they consistently tell each other, resulted in a situation where one of their beliefs could be objectively tested, and they were all wildly wrong in exactly the same way. What does this suggest about all the other stories the mainstream media (CBS exec: "How could Nixon have won? I don't know a single person who voted for him!") tells every day, in exactly the same way, about all sorts of issues that can't be so easily measured? And what does it suggest about the academic institutions that enforce conformity to exactly the same set of opinions and narratives and not only filters information but fires and otherwise silences dissenters? And anyone who suggests that a system so utterly lacking in diversity of thought that there is no one left on campus who will tell them if they're wrong can't be trusted is called "anti-intellectual" and "anti-science". It doesn't tell me that they are wrong about everything, just that they are likely to be just as wrong about some other issues they are adamant about as they were about this one, but that promoting their dogma, not working to find the truth, is what they really care about, which doesn't make them wrong about any particular thing, just not reliable.
sshumaker 10y ago
pcunite - we've discussed in another thread, but the election was actually quite close. HRC had more of the popular vote, and a 1% shift from Trump to Clinton would have resulted in her victory. Here's an excellent analysis from five-thirty-eight: http://fivethirtyeight.com/features/what-a-difference-2-percentage-points-makes/ http://fivethirtyeight.com/features/what-a-difference-2-perc...
- pcunite 10y ago
  I understand what is being reported now. I'm referring to my sense of awareness before the election. It looked like a big win - to me at least.
- fixxer 10y ago
  Sure, but the polls being published and "marketed" had her with an insurmountable lead. Silver had her at 70% and that was generous to Trump compared to many. This isn't about the polls being wrong because the race was tight. Many misrepresented likely turnout (because many are incompetent and thought the 2016 electorate was going to be the 2012 electorate) and the media pumped up those same HRC favorable polls while the pundits attacked the validity of what turned out to be better polls (LA Times). It sure looked like a propaganda machine to me, but I'm just one of those deplorables.
  - grzm 10y ago
    Some honest questions: Do you think it's worth polling at all? Do you think it's possible they made honest mistakes in their models? If you're voting in the election, you're going to have an opinion. Is it possible to work on polling honestly while being an eligible voter? Personally, I think polling is worthwhile. I think it's possible they made honest (and sometimes incompetent, but not malicious) mistakes, and acknowledge that attempting to predict the outcome of a future event is inherently difficult. I think it is possible to do honest work while still voting.
    - fixxer 10y ago
      Polls are useful for the campaign as part of media optimization and field strategy. In terms of value to the electorate, I think polls are anti-value: they are presented to generate response, be it entertainment or motivation. The electorate needs to held accountable for turnout, not tricked into it. I don't believe in requiring voting, but I despise the methods used to GOTV. Either you possess civic engagement or you don't
mikeash 10y ago
It's amusing to see all the people coming out of the woodwork to claim that they knew all along that Trump would win big. The vote was almost dead even. If you though Trump was going to win big then you were just as wrong as the places that predicted a big Clinton win.
- hga 10y ago
  Quibble: since this is done by Electoral College adjusted voting, it wasn't "almost dead even" by that accounting, as of right now the AP called Arizona for Trump, making it 290-228. Michigan leans him, New Hampshire leans Hillary, if those two break in those ways, it'll be 316-232, which is fairly big as these things go, close to Obama's 2012 win. About as big as I expected, although not quite as big as I hoped, then again, there are always surprises in Presidential elections. I didn't expect Trump to get Pennsylvania (or Toomey to win), I expected Russ Feingold to win (yesterday saw a report on the general issue of Team Hillary ignoring/taking for granted the Rust Belt, including ignoring a plea for help from Feingold), and one of the biggest surprises is that Minnesota truly was in play (margin looks to have gone from ~200K in 2012 to ~40K).
  - mikeash 10y ago
    The electoral college margin doesn't mean a whole lot. Tiny swings in the vote can result in huge swings in the electoral college. If you lose your balance on top of a mountain, teeter both ways, then fall to the right, the fact that you might come to rest miles from the peak doesn't mean it wasn't a near thing.
    - NhanH 10y ago
      If Clinton has lost only one of her firewall states, your comment would have applied. The fact that Trump flipped quite a few of them makes it no longer a close one. If two countries play a best-of-5 soccer games, and one beat the other 3-0 with three consecutive 1-0 games in the last minute. It would have been hard to claim that one team isn't obviously better and the margin of winning is small.
PhantomGremlin 10y ago
The major media outlets were looking outright deceptive from my perspective. I had some CNBC shows saved on my Tivo and I recently went back to look at what they were saying. Here's an extreme example of the deception. Just the day before the election CNBC was showing states such as Montana, North Dakota and South Dakota as "leaning" GOP, not "likely" GOP. But here are the results for those states: Montana Trump 57% Clinton 36% North Dakota Trump 64% Clinton 28% South Dakota Trump 61% Clinton 32% From this one could easily conclude that the mainstream media deliberately chose to deceive about the expected outcome in those states. Why? Perhaps to not show as much dark red on the national map? Yes there were true battlegrounds or toss-ups. E.g. Florida only went for Trump by about 1.3%. But there were more than 2x as many Trump voters than Clinton voters in North Dakota. It was a bald-faced brazen lie to call that "leaning".

forkandwait 10y ago

If I wanted to study a book for 10 hours on political polling and the math it takes to model the results, what book would you suggest?

mrlatinos 10y ago
I'm probably not the right person in the company to ask.
dlss 10y ago
The two best resources I'm aware of in this area are: * For designing psychology experiments: https://smile.amazon.com/Conducting-Research-Psychology-Measuring-Weight/dp/0495598194 https://smile.amazon.com/Conducting-Research-Psychology-Meas... * For Bayesian modeling and inference: https://smile.amazon.com/Bayesian-Analysis-Chapman-Statistical-Science/dp/1439840954 https://smile.amazon.com/Bayesian-Analysis-Chapman-Statistic... However, unless you are already very knowledgeable in these subjects, it will take longer than 10 hours to understand them. Very much worth the read though.
- trendia 10y ago
  Depending on his / her statistics knowledge, Gelman could easily take 40 - 100+ hours to really understand.
- forkandwait 10y ago
  Thanks, but that is a somewhat crazy answer. I have a BA in mat with a few upper division stat and prob classes, but desire to do a grad degree in stats (yuck). I just want to read a book about how pollsters practically execute their job.
  - sn9 10y ago
    You can usually find the methodology analysts like 538 use. The better ones are usually pretty transparent about them.

ubernostrum 10y ago

And with good reason: polls measure popular vote, not electoral vote, and Trump most certainly did not win the popular vote by 8 points. He didn't even come close to that margin.

disposable099 10y ago
Well, they shouldn't. The point of polls is to predict who's going to win. It's not that hard to take into account Electoral College anyways.
MichaelBurge 10y ago
I assume you're worried he didn't collect location data? Sometimes you can get the location data from their IP address or zip code(if it's a form).
- ubernostrum 10y ago
  So if this poll said Trump was "8 points ahead", where did it say he was "8 points ahead"? A generic statement of "8 points ahead" is, when hearing about a poll with zero other context read to mean "8 points ahead in a national poll of likely voters with a representative sample", which is a national popular vote measure of limited utility in predicting electoral outcome.
  - mrlatinos 10y ago
    He was 8 points ahead in a national poll of likely voters with a representative sample when asked who they would be more likely to vote for. I don't have specifics besides the 8 pt. lead because I'm not on the analysis side of things and the results weren't published. And I agree, phone polling is fairly limited as far as outcome prediction goes. Part of the reason they didn't publish is because they thought it was a fluke.
    - ubernostrum 10y ago
      So, did he end up with an 8-percentage-point national popular-vote lead? If not, seems the poll was inaccurate.

youarefired 10y ago

That interesting. I run a silly little website website that invited people to vote against one of the two candidates [1]. I watched our analytics like a hawk during the couple of weeks before the elections in order to adjust messaging, content and ads. This, of course, wasn't scientific at all except for the fact that we were running ads to attract both sides of the discussion equally. They were geographically limited to the US. About a week before the elections the trend was unmistakable. The support for Trump in the form of CTR, engagement and votes to the hundreds of different ads out there told a story. We actually tried to get this story out and attempted to publish a press release. It was rejected by all of the top press release agencies we used for other business. Not one of them wanted to publish that things were not looking good from Clinton, even from a silly little entertainment site. To confirm the bias a few days later we issued a press release indicating that Clinton was catching-up and Trump was predicted to lose. Everyone published the release. We can't have the media operate with such bias in this country. Freedom of the press is important, but when the press become political activists we should all yell foul and reign them in. The first thing totalitarian regimes do is take control of the press. We can't have a press that is so far to the left that they won't report the truth, or worst, distort it. I know this is just one data point. There are plenty more from other sources. The bias and manipulation are very real. [1] http://youarefired.com/ http://youarefired.com/

hga 10y ago
We can't have the media operate with such bias in this country. Freedom of the press is important, but when the press become political activists we should all yell foul and reign them in. The first thing totalitarian regimes do is take control of the press. We can't have a press that is so far to the left that they won't report the truth, or worst, distort it. You say we "can't", but we demonstrably do have such a press. And you're 100% correct to discern the totalitarian nature of it. Which is why alternative sources have become so important, and as more people's "Pravda breakers" trip (https://news.ycombinator.com/item?id=12921328 https://news.ycombinator.com/item?id=12921328) the less important this old press becomes. Look no further than the former "Newspaper of Record", it's post-election front page, it's crashing revenue, and why I call it "Carlos Slim's personal blog". Historically it started with Reagan's zapping of the "'Fairness' Doctrine", which allowed right of center talk radio to become a thing, which I'm sure helped prime things for rise of the Internet. Drudge's 1998 reporting of a story spiked by Newsweek (sold in 2010 for a dollar and assumption of its liabilities) clearly marked this was a decisive change ... and yet our "press", MSM, this whole old establishment, seems determined to stay the course. Which, just citing the two financial bits above, looks like a path to oblivion (although many of us are predicting they'll get direct government support soon enough, but that won't get them readers), and therefore self-correcting in, say, the medium term. So I'm not quite so alarmed, as long as the Internet stays free of censorship at the IP and maybe DNS levels.

mikeash 10y ago

Given that your poll was off by 8 points, was that not the correct decision?

freehunter 10y ago
The correct way to present polls is with a sampling of many different polls, like Real Clear Politics or 538. Yes, all the polls were off across the board, but individual polls have a higher likelihood of being wrong than a bunch of polls combined.
mrlatinos 10y ago
Most of our other polls were off as well, but in Clinton's favor, so do we not publish those either? At the time, there was no way to know what was correct, so shouldn't sharing the information be more important than anything else?
- mikeash 10y ago
  They were a lot closer, weren't they? 8 points off seems rather large.