10 ms·
The Rise of the Weaponized AI Propaganda Machine
- SomeStupidPoint 10y agoThis is our fault, you know. Many of us, myself included, have worked on technologies that were meant to profile and manipulate users. Many more of us worked for companies who did this or did similar things through traditional means (eg, psychologically manipulative ads). We've not only allowed, but encouraged technology to have a manipulative and addictive relationship with users. We took the paycheck and played red team against humanity. (This is common -- red team is sexier than blue team, and has an easier time of it, pretty much in any field.) So, personally, I think it's time we spent some of our effort undoing that damage and playing blue team. I don't know what that looks like in full -- though I have some ideas -- but I do know that allowing the weaponization of psychology against humanity to continue unabated will end in disaster. In some ways, that's the struggle of the 21st century: will technology serve the public or enslave it.
- sadgit 10y agoTheres alot of work to do on fleshing out what "blue team" meand and how one can meaningfully contribute to it.
- lazaroclapp 10y agoMaking this networks transparent would be a pretty good first step. So, maybe when you see a sponsored FB post or "news" article it will say who is sponsoring (traversing shell companies on a best effort policy), why is it being targeted to you (at least based on which selectors) and how many variants of the ad exist.
- flukus 10y ago> Making this networks transparent would be a pretty good first step. Making them open and not controlled by corporate interests comes way before this.
- Natsu 10y agoIt would seem more honest to help people avoid manipulation in general, rather than simply helping to manipulate people in a different way.
- SomeStupidPoint 10y ago...which is why that's basically what I suggested. "Red team" and "blue team" are referencing offensive and defensive teams using standard security lingo. In this case, red team is trying to manipulate and control the public. As another commentator pointed out, it's a little unclear what blue team even should be doing. I avoided just saying "help people avoid manipulation", because the implementation of a defensive strategy might be self-directed manipulation overriding the other manipulation you're exposed to, as it's not necessarily possible to avoid manipulation entirely. (Or even a deairable goal, as you might want to adjust your existing habits.) My main point was that we need to start developing defensive tactics and strategies, because the offense dominated present situation will end in catastrophe.
- Natsu 10y agoFair enough. The context you intended wasn't apparent to me before.
- jdoliner 10y agoIt's a bit confusing in this context, I interpreted "blue team" as Democrat team.
- SomeStupidPoint 10y agoI realized that after I read the reply, but couldn't edit. C'est la vie.
- zardo 10y ago>As another commentator pointed out, it's a little unclear what blue team even should be doing. If the primary tools of offence are making use of addictive technology and manipulation, the defencive move is making technology available to help people overcome addiction and give direction to their own lives. As you say, self directed manipulation, and I would add introspection. Maybe FOSS tools for cognitive behavioral therapy.
- cs702 10y agoThe article mentions a "groundbreaking study" from 2013 that a psychologist "spent years developing"... It sounds impressive, until I looked at the actual study[0], which uses simple linear or logistic regression on 100 features obtained by low-rank approximation of an m users × n Facebook-likes matrix via plain-old singular value decomposition (SVD)[1], as shown here: http://www.pnas.org/content/110/15/5802/F1.large.jpg http://www.pnas.org/content/110/15/5802/F1.large.jpg I don't know how anyone could call this "groundbreaking," considering that lots of people in Silicon Valley have been using matrix factorization to predict consumer behavior and make recommendations for a couple of decades, give or take. I stopped reading at that point. [0] http://www.pnas.org/content/110/15/5802.figures-only http://www.pnas.org/content/110/15/5802.figures-only [1] http://www.ams.org/samplings/feature-column/fcarc-svd http://www.ams.org/samplings/feature-column/fcarc-svd
- monk_e_boy 10y agoI don't know much about US elections in regard to facebook and fake news, but I did pay attention during Brexit. Most 'news' was fed to us through newspapers (online and offline) the TV and radio. I didn't see a single fake news site, I saw some crap forwarded on facebook most of which had lots of comments calling it out as crap.
- dreamfactory2 10y agothere were a ton of brexit bots on twitter and in comment sections, they later started working for trump, and are now working for le pen. some stuff about it here - http://heatst.com/world/how-russias-twitter-bots-and-trolls-work-with-donald-trump-campaign-accounts/ http://heatst.com/world/how-russias-twitter-bots-and-trolls-...
- dredmorbius 10y agoResurrecting an inexplicably dead comment: Ithere were a ton of brexit bots on twitter and in comment sections, they later started working for trump, and are now working for le pen. some stuff about it here - https://heatst.com/world/how-russias-twitter-bots-and-trolls-work-with-donald-trump-campaign-accounts/ https://heatst.com/world/how-russias-twitter-bots-and-trolls...
- myowncrapulence 10y agoSilicon Valley spent the last ten years building platforms whose natural end state is digital addiction. In 2016, Trump and his allies hijacked them. I wouldn't call it "hijacking" when that was already the intended purpose of those sites. Just because neo-liberals lost their perceived power (I'm neither liberal nor conservative) in said industries from years of abuse of said systems doesn't make it somehow more nefarious now.
- mirimir 10y ago> From now on, the distinguishing factor between those who win elections and those who lose them will be how a candidate uses that data to refine their machine learning algorithms and automated engagement tactics. Elections in 2018 and 2020 won’t be a contest of ideas, but a battle of automated behavior change.
- navs 10y agoI find it ironic that this article includes the following trackers: - segmentio - intercomio - google analytics - doubleclick.net - facebook - mixpanel
- mirimir 10y agoPrivacy Badger detected 0 potential trackers on this page. These sliders let you control how Privacy Badger handles each one. You shouldn't need to adjust them unless something is broken. The domains below don't appear to be tracking you scout-cdn.s3.amazonaws.com cdn.embedly.com fonts.googleapis.com www.google.com fonts.gstatic.com
- ENTP 10y agoI can confirm navs is correct. Brave Browser indicates blocking of 9 trackers: https://googleads.g.doubleclick.net/pagead/id https://static.doubleclick.net/instream/ad_status.js https://api.segment.io/v1/p https://www.google-analytics.com/analytics.js https://widget.intercom.io/widget/ https://api.mixpanel.com/decide/ https://api.mixpanel.com/track/ https://api.segment.io/v1/t https://api.mixpanel.com/track/
- mirimir 10y agoOK, so why the difference? I'm using Firefox 51.0.1 with Privacy Badger 2017.1.26, NoScript 2.9.5.3 and Adblock Plus 2.8.2. Edit: OK, so I had doubleclick.net, facebook.com, google-analytics.com and mxpnl.com untrusted in NoScript. After trusting them all, Privacy Badger still reports no tracking, but lists these trackers: cdn.embedly.com connect.facebook.net www.google-analytics.com fonts.googleapis.com www.google.com Strange. So is Privacy Badger failing to detect tracking?
- bamboozled 10y agoI'm going to take a wild guess here, It's probably NoScript preventing the tracking code form being loaded before ABP can perform detections?
- lurchpop 10y agoOk, so people living outside major cities in UK voting against centralization of unaccountable power in the EU and unemployed, politically abandoned industrial workers in the US Rust Belt were just a bunch of idiot rubes tricked by click bait headlines. Does it occur o anyone they may have actually had real grievances against globalization? Maybe they were tired of being belittled by metropolitan and coastal elites ignoring those grievances and dismissing them as racists to be ignored.
- DiNovi 10y agoPerhaps those grievances get obscured when people parrot the same nonsense about "elites" that the AI propaganda spouts.
- AbrahamParangi 10y agoIf the options are: 1. Public (online) discourse is vulnerable to manipulation by bad-faith actors (and this manipulation is impactful on a geopolitical scale). 2. The events of the US election and Brexit were symptomatic of unrecognized problems and unheard voices. These ideas can both be true. Frankly, I find the notion that (1) isn't true too far-fetched to believe. It sounds like far too good of a business opportunity. Running ads is a valuable activity not because every person who sees the ad becomes a customer, but because in aggregate x% of viewers will consider a purchase and y% will become good customers. Cambridge Analytica sounds to me like they're just applying the standard techniques of digital content marketing to a relatively unsophisticated realm (political advertisement).
- flukus 10y ago1 has been true for a very long time, long before the internet existed, it's commonly known as astro turfing (https://en.wikipedia.org/wiki/Astroturfing https://en.wikipedia.org/wiki/Astroturfing), the wikipedia articles cites Shakespeare is the first known (albeit fictional) example. The approach was instrumental in preventing action on climate change.
- leereeves 10y ago> Public (online) discourse is vulnerable to manipulation by bad-faith actors (and this manipulation is impactful on a geopolitical scale). This is true, but it's not limited to the right. Just one example, the Washington Post and other liberal papers reported that a Muslim-American Olympian was detained by customs. https://www.washingtonpost.com/news/early-lead/wp/2017/02/09/muslim-american-olympian-ibtihaj-muhammad-says-she-was-detained-by-u-s-customs/#comments https://www.washingtonpost.com/news/early-lead/wp/2017/02/09... The fake part? They never mentioned that it happened in December.
- deleted 10y ago[deleted]
- laxatives 10y agoThis seemed inevitable. People lamented the growth of fake news and niche circles reinforcing extreme attitudes on the internet. Ads exist because they effectively target increasingly small and specific groups of people. Its concerning enough when the motive is profit, but the same techniques can be applied to much greater effect as propaganda. The real concern is when those reinforcing themes are applied to individuals, instead of large swaths of people. Using Facebook, Google, LinkedIn, and other data aggregators exposes you to manipulation. It can be convenient when it shows you relevant ads and the motivation is clear (money). But by the time you realize you're being manipulated for greater causes, like electing officials to power, its far too late to revert those changes.
- dredmorbius 10y agoIt's just another form of advertising. The AdTech world should be delighted And if you think advertisers won't exploit this (and aren't already), well ... that's just precious. Bless your heart. Advertising (and "public relations") are rooted in propaganda. Which happen to be the titles of Edward Bernay's two classic works on the subject: https://archive.org/stream/BernaysEdwardPropagandaEN1928153P./Bernays%2C%20Edward%20-%20Propaganda%20%28EN%2C%201928%2C%20153%20p.%29#page/n0/mode/2up https://archive.org/stream/BernaysEdwardPropagandaEN1928153P... http://www.worldcat.org/title/public-relations/oclc/546945 http://www.worldcat.org/title/public-relations/oclc/546945 And it's always been inherently political. Something the advertising industrial support complex, which includes Google, may not wish to admit. https://archive.org/details/BernaysEdwardL.CrystalizingPublicOpinion1923noOCR https://archive.org/details/BernaysEdwardL.CrystalizingPubli... Sleep well.
- davidivadavid 10y agoThat kind of begs the question, though. If you read Bernays, he doesn't have a negative opinion of public relations and advertising, and clearly lays down why they are in fact a service rendered to the public. It makes as much sense to say: propaganda is rooted in advertising and public relations.
- dredmorbius 10y agoNB: raises the question, not begs. Bernays unabashadly used PR and advertising to vile, vile ends, in his professional heyday. He seems to have had something of a change of heart late in life, to the point that he advocated licensing of PR officials, in marked contrast to those he held earlier (also mentioned in this obit): [I]n an interview in 1991, when he turned 100, he said: "Public relations today is horrible. Any dope, any nitwit, any idiot can call him or herself a public relations practitioner." He said he was still consulting with clients and regarded public relations loftily as a "social science."... Around his 100th birthday, he campaigned unsuccessfully to get legislation passed in Massachusetts and other states that would have required the licensing of public-relations practitioners. http://www.nytimes.com/books/98/08/16/specials/bernays-obit.html http://www.nytimes.com/books/98/08/16/specials/bernays-obit.... Views can change over a lifetime.
- SubiculumCode 10y agoPrivacy is important for democracy.
- spitfire 10y agoIf anyone has a copy of the main paper Cambridge Analytica were using please link here. I know that's vague, but to narrow it down on a previous article there was a picture of an individual reviewing said paper. Someone noted it and linked to the original paper. I made a copy, but deleted it while cleaning things up.
- otaviokz 10y agoSo yet another culprit for the DNP defeat in the last election? I'm definitely not a Trump supporter (I'm not even American), but if I remember correctly the DNP also had a gargantuan apparatus of traditional media and social networking tools working 24/7 to help Hillary win the election. I've seen "Fake news" in both sides of every election I followed (Brazil and US since ever and UK in the last 4 years). The big difference now is that, aside from the fact Trump is a scary individual, this time the "big media" side lost. Not that it is any advantage in the case at hand.
- 3131s 10y agoI believe you mean the DNC. DNP is an organic compound used by bodybuilders to make cellular respiration less efficient and therefore increase metabolism. It's quite dangerous :)
- battlebot 10y agoThis is bullcrap and here's why: A large number of people out there have no social media accounts, they don't read news on the Internet, and they don't give two shits about Trump except they wanted to send a message against the system. Do you really think 62 million people were propagandized into voting for Trump? You're insane if you think that. But go right ahead and keep on believing whatever you want. The real truth is that all news is fake. I sat in the airport a week ago with CNN blaring and I can tell you first hand that Wolf Blitzer is nothing but a propgandist himself. Who is he a messenger for? Beats me, but the quality of what passes for news from all media is extremely low and getting worse. Discounting why people voted for Brexit and for Trump is done at your own extreme peril--but by all means, keep poking a the sleeping giant.
- PretzelFisch 10y ago"Scout combines science fiction and journalism to bring you weekly online dispatches on the future of technology."
- EGreg 10y agoPeople will keep explaining it with regular explanations. But the rise of AI threatens all our systems that rely on the inefficiency of an attacker. The first one to go is trust in the media. Look at facebook and how it uses big data to manipulate people into spending more and clicking more and posting more and checking notifications more. Even if it means echo chambers. Think about your own experience. Are you more addicted today than, say, 5 years ago? People think that they make their own decisions but when the best ideas are aggregated then it's you vs an entire big data server farm crunching numbers. Soon your willpower won't be enough, because peer pressure, media stories and even laws will bring you back to increasing their metrics. They can micromanage more and more interactions, get dossiers on everyone, and they will outmaneuver old media more easily every year.
- dkarapetyan 10y agoSo. Much. Clickbait. Isn't the article itself an example of such "weaponized" propaganda? On a slightly meta point I'm starting to notice that a much larger portion of the population is prone to conspiracy theories than I previously suspected. It might be another one of those broken design things. The propensity to see patterns in everything naturally gives rise to conspiracy theory galore.
- zeteo 10y ago> By leveraging automated emotional manipulation alongside swarms of bots, Facebook dark posts, A/B testing, and fake news networks, a company called Cambridge Analytica has activated an invisible machine that preys on the personalities of individual voters to create large shifts in public opinion And somehow managed to overcome the constant "fact checking" by major cable networks, the Post, NYT and generally a large majority of the established media. Let's be serious. The narrative is ridiculous, but even if we were to take it at face value the cure is worse than the disease. Basically voters can no longer be trusted with making up their own mind in a democracy and they need to be automatically manipulated for the greater good: > From now on, the distinguishing factor between those who win elections and those who lose them will be how a candidate uses that data to refine their machine learning algorithms and automated engagement tactics. Elections in 2018 and 2020 won’t be a contest of ideas, but a battle of automated behavior change. I think at this point the losing side needs to accept that they didn't make their case properly ("the other guy is worse" only gets you so far) and stop chasing ridiculous scapegoats. American democracy has endured through secession, yellow journalism and Joseph McCarthy. It will survive Facebook and sentiment analysis bots.
- pgodzin 10y ago> And somehow managed to overcome the constant "fact checking" by major cable networks, the Post, NYT and generally a large majority of the established media. Well a big part of the campaign was delegitimizing the establishment as "fake news" that is in bed with Hillary. Combined with a bombardment of headlines that confirm their biases, targeted by personality, it could have a big impact.
- flukus 10y agoCould it be that the tactic only worked because the establishment was in bed with Hillary? She is about as establishment as the come after all.
- pgodzin 10y agoExcept the delegitimizing was done to the point where fact checking actual lies and falsehoods were simply ignored. This targeted approach didn't have to "overcome" that, as the parent claimed, just make people ignore it entirely.
- malloryerik 10y agoCambridgeAnalytica's site is interesting. https://cambridgeanalytica.org https://cambridgeanalytica.org Overall this kind of thing very much jibes with Yuval Harari's ideas about liberalism's approaching demise, as forecast in his book Homo Deus. Edit: A Bloomberg article of note here https://www.bloomberg.com/view/articles/2016-12-08/no-big-data-didn-t-win-the-u-s-election https://www.bloomberg.com/view/articles/2016-12-08/no-big-da...
- lend000 10y agoThis is painfully naïve, as though the careful selection and omission of stories to craft narratives on the more popular "real news" sources is somehow less concerning.
- jawarner 10y agoAn election is a war of opinion, and whoever wields the strongest memes wins. Ten times as many people subscribe to Trump's subreddit as Clinton's, and Trump memes circulate other anonymous Internet culture hubs like 4chan. At stake is the position of the most powerful man in the world. It has been shown to be possible that a group of skilled programmers / hackers could influence the election. Why wouldn't, say, an expert team be hired to program ML that optimizes a candidate's size of web presence? Cheap price for a lot of political power. A "weaponized propoganda machine"? Maybe not. But if it's really that easy to game the system, you can't ignore the possibility.
- deleted 10y ago[deleted]
- chatmasta 10y agoWhen Obama hires "data crunchers" [0] to "rally individual voters" [1], it's "innovative microtargeting." When Trump does the same, with four years of technology advancement, it's "weaponized AI propaganda." C'mon... [0] http://swampland.time.com/2012/11/07/inside-the-secret-world-of-quants-and-data-crunchers-who-helped-obama-win/ http://swampland.time.com/2012/11/07/inside-the-secret-world... [1] https://www.technologyreview.com/s/509026/how-obamas-team-used-big-data-to-rally-voters/ https://www.technologyreview.com/s/509026/how-obamas-team-us...
- uniclaude 10y agoThose two wordings aren't even coming from the same source, what exactly are you trying to prove here?
- chatmasta 10y agoI'm comparing interpretations of what amounts to the same action by Trump and Obama. Admittedly I'm generalizing, but the analogy is more valid than a comparison of apples to oranges. Both Trump and Obama outsourced campaign analytics to professionals, and in both cases there were articles explaining the innovation. In Trump's case, this article (and others I have seen), has a distinctly negative tone. For Obama, the articles praised his data scientists and boasted of their high post-campaign salaries. I challenge you to find an article describing Obama's campaign analytics with similarly negative tone to this article about "weaponized AI propaganda." The differing interpretations of two similar scenarios are worth observing, since the variability represents evidence of bias on the part of the journalists writing the articles.
- webmaven 10y agoThere is a difference in quality in using analytics (whether demographic, psychological, or what have you) to target a state, county, block, or individual with a campaign message (clearly marked as such) in order to try and sway voters to some action, and using the same or similar techniques to create targeted "news" articles not labelled as campaign materials. If even a fraction of what this article is alleging is true, the FEC may have quite a bit to say about it: http://www.fec.gov/pages/brochures/notices.shtml http://www.fec.gov/pages/brochures/notices.shtml