4 ms·

This is what I consider/fear the next 'step' for 'fake news' and weaponised propaganda. Giving different audiences different content, and using those difference

by CM30 8y ago

This is what I consider/fear the next 'step' for 'fake news' and weaponised propaganda. Giving different audiences different content, and using those differences to turn them against each other. Picture this: You've got a story which is seemingly very controversial, and will rile up one side of the political spectrum. However, this story is only shown for say, 10-20% of your readers/viewers, and the rest get an innoculous story, or one that says the complete opposite. Now the ones affected by it will post it on social media sites and forums, saying that this is proof of how 'evil' the other side is. But the others there can't see the same story, and will think said users are trolling them. After seeing them keep posting comments about how some minority group is evil with links to innocent stories about kittens and puppies, said users get banned. This causes said users to think they're being targeted by the other side, and then every story they see on the site is designed to build off that. Make the forum/social media/news site owners out to be illuminati pawns or what not and their banning policies a way to stifle dissent. Voila, the site has now basically weaponised A/B testing, and created a localised echo chamber to gaslight its readers into more and more extreme mindsets. That's basically the nightmare scenario. Imagine an enemy state like Russia or China using it against the alt-right/antifa/whatever. Or activists using it to recruit followers. Worse still, imagine it with compromised browser extensions as well. Imagine if a foreign government managed to do this in a way that could let them edit the text on fact checking sites like Snopes or major media outlets like the BBC or CNN or what not. Now that could have some worrying consequences. Still, it wouldn't be impossible to fix/detect. Set up sites like Reddit/Hacker News to automatically archive pages linked to, and send users to the archive rather than the original unless they choose to click through. Avoid using any browser extensions at all, with hosts file blocking for things like ads rather than Adblock/uBlock/whatever. Ask others to double check what every page you see says to make sure it hasn't been compromised/doesn't show different things. Do that and you can at least minimise this style of dissent sowing.

dragonwriter 8y ago
> This is what I consider/fear the next 'step' for 'fake news' and weaponised propaganda. Giving different audiences different content, and using those differences to turn them against each other. That's not the “next step”, it's an established practice. Notably, the recent Russian propaganda effort directed at destabilizing the US via false flag social media accounts and other channels was noted for doing exactly this. (They didn't invent the general concepts either, it's much older.) The actual mechanism in your example would be a bit novel, but it's also fairly easily accidentally penetrated, because real humans aren't 1:1 to online personalities.
- TheOtherHobbes 8y ago
  This is literally how the Brexit vote was won. Cambridge Analytica used personal profiles scraped from FB and (allegedly) from insurance company customer details to run ads that were precisely tailored - almost down to specific users. It only took a small swing to push the vote over the line.
  - isostatic 8y ago
    > It only took a small swing to push the vote over the line. With modern elections results often come down to a tiny tipping point. In the UK 2017 election, tipping the right 400 voters one way or another changes the makeup of government from a DUP backed coalition to a Tory majority[0]. Tipping the other way, and denying a DUP/Conservative coalition, would have taken the right 750 to flip. Turnout was 32.2 million, and the entire election came down to 1200 voters, 1 in 27,000. In 2016 if Clinton had flipped about 60,000 votes (the right ones), or about 1 in 2,300, she'd be president. By comparrison, brexit required a massive swing - 1 in 26 voters. [0] https://www.telegraph.co.uk/news/2017/06/12/theresa-may-just-400-votes-short-majority/ https://www.telegraph.co.uk/news/2017/06/12/theresa-may-just...
  - nradov 8y ago
    You really have no idea whether people voted a certain way because of those ads or in spite of those ads.
    - barrow-rider 8y ago
      Marketing works, we've had 100+ years to tweak and it justify marketing's huge budgets. Tweaked ads work too. Further more, why can't it be both, because of AND in spite of those ads? Successful propaganda often uses false images from another perspective to generate scorn, e.g. obviously idiotic comments from tumblr SJWs (r/tumblrinaction, for example) who are likely sockpuppets. Regardless, marketing is about numbers on the whole, not whether Bob or Sheela specifically voted -- and on the whole it worked for Brexit.
tyingq 8y ago
Hacking into one or more of the big CDN services would make this scale pretty broadly.
AnaniasAnanas 8y ago
Buzzfeed already does that, though I presume that their objective is to damage control as quickly as possible if an article generates a lot of negative responses rather than cause chaos. See https://en.wikipedia.org/wiki/Wikipedia:Identifying_reliable_sources/Perennial_sources#Sources https://en.wikipedia.org/wiki/Wikipedia:Identifying_reliable... > BuzzFeed may use A/B testing for new articles, which may cause article content to change This is actually one of the reasons that make things like git repos (given a certain commit hash), torrents, and ipfs (except IPNS/MFS) extremely attractive - nobody will be able to silently change things afterwards.
pishpash 8y ago
Now you understand why the internet is censored and sanitized in countries that already don't believe in democracy as a viable form of government. The experiment on democracy is entering its most serious test. Can it survive?
- goldfeld 8y ago
  Please, democracy has barely, barely survived the cold war all the world 'round, it's a dead horse to beat.
andrewflnr 8y ago
I think people will catch on faster than you think. People are (somehow) already in the habit of screenshotting tweets and sharing the image. Person A will quote pieces of the article to rip it, person B will post a screenshot of the article saying something different, person A will post their own screenshot, and it will unravel.
- kps 8y ago
  Encrypted Media Extensions will ‘fix’ that.

by CM30 8y ago

dragonwriter 8y ago

> This is what I consider/fear the next 'step' for 'fake news' and weaponised propaganda. Giving different audiences different content, and using those differences to turn them against each other. That's not the “next step”, it's an established practice. Notably, the recent Russian propaganda effort directed at destabilizing the US via false flag social media accounts and other channels was noted for doing exactly this. (They didn't invent the general concepts either, it's much older.) The actual mechanism in your example would be a bit novel, but it's also fairly easily accidentally penetrated, because real humans aren't 1:1 to online personalities.

TheOtherHobbes 8y ago
This is literally how the Brexit vote was won. Cambridge Analytica used personal profiles scraped from FB and (allegedly) from insurance company customer details to run ads that were precisely tailored - almost down to specific users. It only took a small swing to push the vote over the line.
- isostatic 8y ago
  > It only took a small swing to push the vote over the line. With modern elections results often come down to a tiny tipping point. In the UK 2017 election, tipping the right 400 voters one way or another changes the makeup of government from a DUP backed coalition to a Tory majority[0]. Tipping the other way, and denying a DUP/Conservative coalition, would have taken the right 750 to flip. Turnout was 32.2 million, and the entire election came down to 1200 voters, 1 in 27,000. In 2016 if Clinton had flipped about 60,000 votes (the right ones), or about 1 in 2,300, she'd be president. By comparrison, brexit required a massive swing - 1 in 26 voters. [0] https://www.telegraph.co.uk/news/2017/06/12/theresa-may-just-400-votes-short-majority/ https://www.telegraph.co.uk/news/2017/06/12/theresa-may-just...
- nradov 8y ago
  You really have no idea whether people voted a certain way because of those ads or in spite of those ads.
  - barrow-rider 8y ago
    Marketing works, we've had 100+ years to tweak and it justify marketing's huge budgets. Tweaked ads work too. Further more, why can't it be both, because of AND in spite of those ads? Successful propaganda often uses false images from another perspective to generate scorn, e.g. obviously idiotic comments from tumblr SJWs (r/tumblrinaction, for example) who are likely sockpuppets. Regardless, marketing is about numbers on the whole, not whether Bob or Sheela specifically voted -- and on the whole it worked for Brexit.

tyingq 8y ago

Hacking into one or more of the big CDN services would make this scale pretty broadly.

AnaniasAnanas 8y ago

Buzzfeed already does that, though I presume that their objective is to damage control as quickly as possible if an article generates a lot of negative responses rather than cause chaos. See https://en.wikipedia.org/wiki/Wikipedia:Identifying_reliable_sources/Perennial_sources#Sources https://en.wikipedia.org/wiki/Wikipedia:Identifying_reliable... > BuzzFeed may use A/B testing for new articles, which may cause article content to change This is actually one of the reasons that make things like git repos (given a certain commit hash), torrents, and ipfs (except IPNS/MFS) extremely attractive - nobody will be able to silently change things afterwards.

pishpash 8y ago

Now you understand why the internet is censored and sanitized in countries that already don't believe in democracy as a viable form of government. The experiment on democracy is entering its most serious test. Can it survive?

goldfeld 8y ago
Please, democracy has barely, barely survived the cold war all the world 'round, it's a dead horse to beat.

andrewflnr 8y ago

I think people will catch on faster than you think. People are (somehow) already in the habit of screenshotting tweets and sharing the image. Person A will quote pieces of the article to rip it, person B will post a screenshot of the article saying something different, person A will post their own screenshot, and it will unravel.

kps 8y ago
Encrypted Media Extensions will ‘fix’ that.