10 ms·
Inside the Secret World of the Data Crunchers Who Helped Obama Win
- plinkplonk 14y agoDoes anyone have any info about the tech stack?
- spamizbad 14y agoI don't know about their data scientists, but I've heard from people who know engineers on the campaign that they're essentially a Python shop. Not sure about the rest of the stack...
- abecedarius 14y agoA more interesting question, perhaps: what data were they mining?
- patrickgzill 14y agoHilarious! What gave PA to Obama was the blocking by the courts of the Voter ID act that the legislature passed. Note that in all the praise about individual contibutors in the article, there is no mention of "bundling" and the routine use of cutouts, both rather unsavory practices that both sides use. EDIT: this site may prove a useful antidote to the above article: http://www.washingtonpost.com/wp-srv/special/politics/campaign-finance/?tid=rr_mod http://www.washingtonpost.com/wp-srv/special/politics/campai... Romney raised more than Obama in total $$$. Even without the SuperPAC contributions that gave Romney more money, the two campaigns were within about 10% of each other in terms of what they raised.
- sethg 14y agoObama took PA by five percentage points, which is consistent with his margin in the state-level polls that were taken throughout October. In absolute numbers rather than percentages, his margin is about 300,000 votes. Are you really trying to argue that a voter ID law would have taken enough people out of the electorate to flip the state?
- patrickgzill 14y agoYes. Fraud in Philly voting is legendary and almost on par with Chicago. I can't find the registered voter turnout for 2012 yet, but in past years Philly RV turnout has been as high as 108%.
- kirse 14y agoAhh that's Philly for ya: http://www.foxnews.com/politics/2012/11/06/judge-issuing-order-to-reinstate-booted-philadelphia-election-officials/ http://www.foxnews.com/politics/2012/11/06/judge-issuing-ord... Unfortunately, it happens everywhere and from both sides.
- jivatmanx 14y agoExcellent John Stewart segment on the issue of voting fraud and these voter ID laws. For the most part, not a single example of fraud was found, so it's hard to justify laws that disenfranchise large segments of society. http://www.huffingtonpost.com/2012/10/03/jon-stewart-slams-voter-id-laws-nonexistent-in-person-fraud_n_1935344.html http://www.huffingtonpost.com/2012/10/03/jon-stewart-slams-v...
- waterlesscloud 14y agoI can buy that voter fraud is pretty limited. But it's simply untrue that there's not a single example. People are convicted and go to jail for it. Not many, but some. Saying they don't seriously undermines the credibility of the argument.
- peteforde 14y agoAllocating massive resources to stamping out a statistically insignificant issue (given that we have no intelligence on who those votes are for, the end result is moot) makes it hard to take Republican voter ID frenzy at face value. Those strategists are smart and they know exactly what happens if you make it harder for the down-trodden to vote. Moderate, intelligent Republicans should be embarrassed by Voter ID. Is this the small government you keep referring to?
- brown9-2 14y agoI'm confused on how the point about fundraising is relevant to your other points.
- patrickgzill 14y agoCompare the hyperbole in the article, breathlessly talking about the geniuses on the campaign - then compare with the hard numbers compiled in the link I posted.
- brown9-2 14y agoThe amount of money raised is not the only metric for measuring how smart of successful a campaign is. I don't think it's even a good metric.
- downandout 14y agoThe gist of this article is that Obama was able to use a combination of data mining and creative marketing to pinpoint impressionable voters in crucial areas with a view toward manipulating the electoral college system to his advantage. This is yet another reason to toss the electoral system in the garbage and set it on fire. In no event should a relatively small subset of the population in a handful of states be responsible for choosing the President of the United States.
- andrewcooke 14y agothe electoral college system only constrains the location of targets. without it, you might make the problem harder, but you would not make it impossible.
- btilly 14y agoNot even that constrained. They had a lot of fundraising targeting as well, and that was nationwide.
- jivatmanx 14y agoHad we used a popular voting system rather than electoral college system, this system would have been expanded to every state in the nation, and Obama would likely have won by a far wider margin than he did. Again, this system was about identifying impressionable voters and marketing to them properly. The location of swing states is simple and public information, not proprietary.
- downandout 14y ago"Had we used a popular voting system rather than electoral college system....Obama would likely have won by a far wider margin than he did." That is quite debatable. There is no way of knowing how many people simply don't show up to the polls because they live in a state that leans heavily toward the opposition and therefore know their vote will have exactly zero effect on the election.
- soupboy 14y agohttp://www.slate.com/articles/news_and_politics/victory_lab/2012/01/project_dreamcatcher_how_cutting_edge_text_analytics_can_help_the_obama_campaign_determine_voters_hopes_and_fears_.html http://www.slate.com/articles/news_and_politics/victory_lab/... - Slate article from Jan 2012 "Project Dreamcatcher How cutting-edge text analytics can help the Obama campaign determine voters’ hopes and fears."
- brown9-2 14y agoA similar profile from Mother Jones pre-election: http://www.motherjones.com/politics/2012/10/harper-reed-obama-campaign-microtargeting?src=longreads http://www.motherjones.com/politics/2012/10/harper-reed-obam...
- chockablock 14y agoThanks! This is much better than the article in the OP. It also links to a number of nice other pieces on the topic, including 2 Slate articles: http://www.slate.com/articles/news_and_politics/victory_lab/2012/02/project_narwhal_how_a_top_secret_obama_campaign_program_could_change_the_2012_race_.html http://www.slate.com/articles/news_and_politics/victory_lab/... http://www.slate.com/articles/news_and_politics/victory_lab/2012/01/project_dreamcatcher_how_cutting_edge_text_analytics_can_help_the_obama_campaign_determine_voters_hopes_and_fears_.single.html http://www.slate.com/articles/news_and_politics/victory_lab/...
- ryanwaggoner 14y agoI find this really interesting, but I'm unclear on the proof that this made a big difference for Obama's campaign success. I'm sure Romney had people crunching the numbers as well, and he actually raised more money than Obama. This was a pretty narrow victory, there are a lot of structural, ideological, and demographic biases at work here, and Obama did worse among most groups than he did in 2008. Put another way, is there any evidence that Obama would have lost or even done much worse without this data crunching effort?
- d23 14y agoIt may have been a narrow victory in the popular vote, but it was an absolute stomp in the electoral college. Obviously we can't go back in time and try it a different way so you're never going to have any "proof", but there was clearly a strategic advantage by the Obama camp in the battleground states.
- ajross 14y agoThere's at least some evidence that, in fact, the Romney campaign fooled itself into believing the "unskewed" polling models which simply (and incorrectly) disbelieved the demographic makeup of the polls. So yeah, it's hard to believe the republican campaign could have run a heavily data-driven campaign. And the evidence that Obama would have actually lost is obviously not going to turn up. But you can look at the relative performance in "swing" states vs. the general electorate for hints. He clearly outperformed among the demographics that ended up deciding the election, as witnessed by the very large electoral victory on the basis of a very narrow (currently <2% I believe) popular vote margin.
- ryanwaggoner 14y agoThere's at least some evidence that, in fact, the Romney campaign fooled itself into believing the "unskewed" polling models which simply (and incorrectly) disbelieved the demographic makeup of the polls. So yeah, it's hard to believe the republican campaign could have run a heavily data-driven campaign. Sorry, what is that evidence? The fact that they were saying they thought they would win? The fact that they didn't give up? Did you really expect them to? It's important not to confuse the public behavior of a campaign with their internal beliefs. And Romney really could have won this, had things gone a little differently.
- lukethomas 14y agoIt's interesting that they mention email lists - I tried unsubscribing several times from the President's list, and it never worked. Anyone else notice this?
- zeteo 14y agoI think polling on such a detailed level is fundamentally changing the democratic process. In the past, politicians had to be leaders - present a vision that was quite possibly unpopular to start with, and persuade people to come around to it. (See e.g. JFK and the space program.) Nowadays they're becoming expert hagglers: keep this mini-group barely content, say exactly the right nice things to the other, ignore a third altogether because they're 10% over in enemy territory. I don't know where this is leading to, but it's definitely a big change.
- herge 14y agoThe oposite opinion can also be argued, that now politicians are a lot more connected to the populace, and that the will of the people is now represented much better than it was in the past.
- rmk2 14y agoHow does a politician become "more connected" to the populace if said populace is only encountered as an integral function of a wider statistically significant effect? "Dear Candidate, you will have to see people in that part of the country, not because you care, but because our data has shown that a similar visit before was statistically significant enough to matter." On top of that, this data-driven approach actively undermines any connection by reducing it to a functional one. There isn't even the pretence anymore that the candidate cares for the given group due to intrinsic values or some sort of connection other than statistics.
- ceol 14y ago> "Dear Candidate, you will have to see people in that part of the country, not because you care, but because our data has shown that a similar visit before was statistically significant enough to matter." When has a candidate ever campaigned in a place for any other reason other than an attempt to snag votes? The advent of computer-driven data didn't suddenly steal the innocence out of campaigning.
- fusiongyro 14y ago
- RyanMcGreal 14y agoInteresting. I somehow got added to the Obama campaign mailing list, and their recent fundraising emails politely noted that their records did not show me having donated any money (I'm not American and don't live in the US). They also said that their records might not reflect my donation if I had made it through a different channel. However, this article suggests that all the channels go to the same database. I wonder whether one of the following scenarios is correct: * The database has eventual consistency (maybe my donation through channel X would take some time to be reflected by the campaign team); * The message was a ruse to throw off independent and Republican analysts to the sophistication of the database; * It was CYA in case something had gone wrong.
- brudgers 14y agoJust for clarity, political contributions directly to a political campaign are a matter of public record in the US. Thus this piece of knowledge was not the result of new data mining techniques, it's been used for a number of years.
- snowwrestler 14y agoThey are only matters of public record if the total donation amount exceeds a limit, I think $200.
- jychang 14y agoCYA is most likely, although #2 is possible. Changing names (due to marriage, for example), moving, losing (and thus getting a new SSN) are all possible, and might screw up the database. A small CYA phrase helps a lot.
- binarray2000 14y agoFascinating stuff... and something I want to learn more about. Can you please recommend me books/software applications/online courses for doing the work these guys were doing for Obama campaign? (Just to clarify: I am a developer and I grok SQL. So that part is covered. Other parts... less so.)
- peteforde 14y agoI recommend Collective Intelligence, published by O'Reilly. If you look online there's Ruby source code examples to go with the Python in the book.
- clarkmoody 14y agoWhat this article seems to show is that the election was not about political ideology. It was about getting out the vote. The Obama campaign fine-tuned its message to scientifically increase returns in those targeted groups. The Romney campaign played the 'smoke-filled room' game (for all we can tell). In none of the presidential debates did either man speak in depth about his governing philosophy (or the Fed for that matter). Their talking points rang loud and clear over and over, and finger-pointing abounded, but neither spoke of his core convictions. The campaign speeches were much of the same. For all of the complaints about Romney's lack of details in his economic plan, the President didn't do any better, simply offering a 'stay the course' message (he learned something from Bush). Therefore, given this article and the lack of ideology involved in the campaigns, I conclude that the voters were not persuaded by philosophy. They were targeted by the campaign so that the powers that be could keep their power. Looking back to the electoral college results of 1980 and 1984 shows what a strong, clear ideological message can do. Those familiar with Reagan's speeches will know why. So for those who say that America has embraced any certain ideology, I say that it has not, since ideology hasn't been on the ballot for a long time. People have embraced a man, and the philosophical debate has reached a new low.
- hackinthebochs 14y agoI don't know what campaign you were watching, but I heard a lot about governing philosophy. Abundantly from the Democrats side, but also from the Republicans. Of course, they won't go into minute detail about policies on the national level; most people simply don't want to hear that. But it seems clear to me that Obama's political philosophy resonated with more people than Romney's.
- forensic 14y agoITT we confuse talking points and identity politics with PHILOSOPHY. Jesus. You know the intellectual landscape is bleak when even hackers are willing to entertain the notion that democracy/mob rule has anything to do with philosophy, or that politicians operate in line with some sort of rational ethic other than straight forward power broking.
- harshaw 14y agoI'd sure like to know more about their methods and algorithms. What exactly "is" big data is glossed over so much that I can't tell if they are doing something unique here or simply applying a large degree of automation.
- wilfra 14y agoToo bad the single biggest piece of data here is going to be ignored. If they started competing with Silicon Valley for talent and put thousands of these guys to work in a building somewhere to fix this country, they'd quickly become the 'nuclear codes' of the USA. But they wont bother, they'll immediately forget all of this until the next campaign and go back to politics as usual.
- f137 14y ago"A politician thinks about the next elections — the statesman thinks about the next generations." There are no leaders anymore.
- azernik 14y agoNate Silver made an interesting point a while back when he compared the political data revolution to the Sabremetrics revolution in baseball. In baseball, the journalists and pundits were way ahead of the professional operators in moving from gut calls to data-crunching, while in politics, it's the opposite - what Silver and other public number-based forecasters do is commonplace inside the campaigns.
- andrewkkirk 14y agoWant a future in campaign management? Cut your chops with an e-commerce company. While there learn big data, how to test, and read into behavioral economics, and you're set.
- fatbat 14y agoAnyone have additional insights to what is meant here? "...the campaign’s Quick Donate program, which allowed repeat giving online or via text message without having to re-enter credit-card information, gave about four times as much as other donors. So the program was expanded and incentivized." - How did the Quick Donate program work? Like Amazon's 1-Click checkout? - What was the call to action for Quick Donating again? - How was the program expanded/incentivized?