9 ms·

State-backed manipulation rampant on social media

162 points by docdeek 6y ago

api 6y ago
That is an issue, but the real fundamental issue is algorithmic timelines and recommendations that bias content to maximize engagement. What maximizes engagement? Hate, fear, and bombast.
- im3w1l 6y ago
  https://en.wikipedia.org/wiki/List_of_most-liked_tweets https://en.wikipedia.org/wiki/List_of_most-liked_tweets https://en.wikipedia.org/wiki/List_of_most-retweeted_tweets https://en.wikipedia.org/wiki/List_of_most-retweeted_tweets Doesn't seem like you are right about that. Edit: Going by replies might yield a different more negative result, the word "ratiod" imply that many see a reply almost as a downvote. But I was not able to find a list of most replied-to tweets.
  - true_religion 6y ago
    According to that list, K-pop maximizes engagement so Twitter should pivot to being MTV online.
  - hshshs2 6y ago
    replies are only similar to downvotes in the absence of likes, that’s why it’s “ratiod” and not “replied”
  - ben_w 6y ago
    Interesting data — and heartwarming to see! — but I’m not sure it’s the right measure. A thousand tweets that are each seen by a different (non-intersecting) group of a thousand people, each tweet being an accusation of evil directed at a specific person or group, does as much damage as one tweet seen by a million. I hope that this measure would show the same result as the two lists you linked to; my current guess is that if someone did this analysis, the result would be almost as positive, but not quite, on the grounds that hate-inducing tweets are against the TOS and will be hidden or deleted when they reach a visibility threshold.
    - AnimalMuppet 6y ago
      A thousand tweets, each seen by a different, overlapping group of a thousand people, reaches fewer total people, but to those people it looks like multiple independent sources confirming the same information. That is, the information looks more trustworthy. (That was one of the points of the Soviet "active measures" propaganda during the Cold War. They not only had western sources giving the Soviet line, they had multiple sources giving the same line.)
      - ben_w 6y ago
        Thanks for the insight, I thought that might be the case but didn’t know. (I also felt it would be an Apples-to-Oranges comparison when the alternative was a single account tweeting to a million followers).
  - dahfizz 6y ago
    I don't think looking at the 30 most liked / retweeted tweets is a good measure for deciding what kind of content is most engaging. BTS makes up more than half of both of those lists - that's just a list of popular twitter accounts. The most tweeted about people on twitter are Trump, Biden, and George Floyd[1]. I think that gives a pretty good idea of the content occupying people's minds. [1] https://blog.twitter.com/en_us/topics/insights/2020/spending-2020-together-on-twitter.html https://blog.twitter.com/en_us/topics/insights/2020/spending...
- w_t_payne 6y ago
  This is the fundamental design flaw of the system. Fixing it isn't going to be easy, though. Look at the amount of effort that the scientific community tries to put into improving the quality of communication:- peer review, standards for content ... and there's still nagging issues with reproducing results and even outright fraud in some cases. Whatever solution we come up with will likely increase friction in the user experience, but there's lots of room for innovation here.
  - api 6y ago
    Fixing this isn't a matter of adding complexity but of removing it. When timelines were sequential or user-moderated and YouTube was just a search engine and host for videos it was fine. Algorithmic prioritization algorithms were introduced around the 2008-2012 time frame and all Internet discourse went to hell shortly afterwords. What we did was to insert a biasing function between all people that biased content for engagement. What maximizes engagement? Hate, fear, and conflict.
  - AnimalMuppet 6y ago
    The fundamental flaw is human nature. That's not going to be easy to fix.
Dem_Boys 6y ago
Since the first day I meddled in creating online bots I've had the intuition that large scale manipulation was occurring. If a single developer can create a twitter bot in 4 hours that likes posts about a topic, then what could a large company, political party, or entire country do? I've always thought "Well they're just manipulating online opinions" but the capital and blm riots have shown that online opinions can have real world impacts. Family and friends often are distressed by the state of the world, where their view of the world is social media. But how much of social media is real? How many of the radical opinions expressed are made by real people? Of those real people, how many of them were influenced by bots or "computational propaganda"? I believe that we, even really smart engineers, are easier to influence that we want to believe
- octocop 6y ago
  > Family and friends often are distressed by the state of the world I see this a lot too, and I have seen it the other way around as well. Where people believe everything is either Armageddon or a non-issue. Ironically I think this effect is applied to itself. Are we distressed because we think our world-view is wrong so we seek to the extremes for validation?
  - Dem_Boys 6y ago
    I'm no expert but this is my opinion. 1. We are all products of our environment 2. The internet and social media is becoming a higher percentage of our environment every day 3. Large parts of the internet and social media makes money by occupying our attention 4. The best way(that we know of) to occupy our attention for long periods is through hyperbolic/enraging material ^^^ Doing this long enough leads to enraged/hyperbolic population and I don't know of a reasonable way to stop it :(
    - shaolinspirit 6y ago
      i would disagree with #1, we are products of our curation and observation of information that is coming in forms of feelings or outer causes. you can be in a bad environment but still curate where do you put your attention by explicitly acknowledging what subjective values do you carry during your lifetime. as above, curation is the main skill of the future, you can use youtube to watch stupid cat videos or to watch self development content, it is up to the viewer. if you are willing to read lies, you deserve to be deceived. with the current abundance of information and possibilities it is a blasphemy to argue that people are manipulated, they are rather choosing this by being lazy or not curious enough or because it already confirms their biases / beliefs. it is sad to see that people replicated the TV (Tell-Me-Vision) model on the internet.
mnw21cam 6y ago
Surely we can widen the issue from state-backed manipulation to lots-of-money-backed manipulation.
- BrandonMarc 6y ago
  ... and the social media giants have lots of money, and are in the perfect position to manipulate. Motivation and opportunity ...
- Miner49er 6y ago
  And the article's suggested solution is: > These companies should therefore increase their efforts to flag and remove disinformation, along with all cyber troop accounts which are used to spread harmful content online. So the suggested solution is to have different lots-of-money companies stop the lots-of-money manipulation. Of course, if you are doing the right kind of manipulation (ads), that's allowed. Seems a bit hypocritical to me.
- mindslight 6y ago
  Lots-of-money-backed manipulation is how things have been traditionally. "Never pick a fight with someone who buys ink by the barrel". It tends to be a stabilizing force for society, despite the malevolence (eg Iraq war). Meanwhile state-backed (or alternatively, lots-of-foreign-money-backed) seems to be a destabilizing force. We've presently got two large political movements, both essentially protesting against out of control government, but yet they've ended up being each other's enemy.
  - mnw21cam 6y ago
    I don't think the distinction between state-backed or lots-of-money-backed manipulation is really what makes it stabilising or destabilising. Rather, I think it whether the manipulation separates out the manipulated into an echo-chamber of similarly-minded outrage or not. For example, a government statement based on scientific investigation, such as "Covid-19 is a dangerous disease, but it helps if you wear a mask" is state-backed manipulation, but it doesn't try to isolate the manipulated into an echo-chamber, and it is intending to be stabilising. Conversely, some of the worst parts of social media can whip up a storm of outrage and hatred because the algorithm (which is intended to maximise the advertising profits of a commercial company) recommends outrageous stories that make you angry, but that you are likely to agree with. This is lots-of-money-backed manipulation, but it does isolate people into echo-chambers, and it is destabilising. Regarding having two large political movements that have become enemies, I think this is an emergent property of a complex interaction between politics and the tendency of people to get sucked into echo-chambers that (as an emergent property, not an intended outcome) steer people towards thinking that the other side is the devil incarnate. Political parties will always disagree with each other. I admire politicians who can say to each other that they disagree strongly with the other's point of view, but they respect them as a person and get on with them well. I think it will be very hard to destroy the natural tendency of social media to form echo-chambers, but it should be attempted. Regarding politics, I think it's interesting that countries that have election systems that discourage two-party politics in favour of multiple parties having to make compromises with each other tend to be more stable and friendly about it.
VBprogrammer 6y ago
To me the biggest issue that was raised by the Trump bans is that we shouldn't be relying on these channels for communication with our governments and elected representatives. Boris Johnson had a brief flirtation with Facebook Q&A sessions. Our governments should broadcast any messaging in the widest possible sense. Official government websites, live feeds to anyone who cares for it etc.
- krapp 6y ago
  The only reason this was ever an issue is that Donald Trump chose to use his personal Twitter account as his primary means of communication, to the point that it had to be recognized as a de facto official channel. And that's mostly because he believed the press and every other avenue was controlled by leftists. This is far more a "Donald Trump is a paranoiac" problem than a social media problem per se. Nothing wrong with a politican doing an AMA on Reddit or a Facebook Q&A or having a Twitter account, which they all do at this point. Politicians using social media as a convenient way to communicate with people directly can be a net benefit for society, although probably more so the more local government becomes. But using social media exclusive of other, official channels is obviously a problem.
- caseysoftware 6y ago
  Twitter, Facebook, YouTube, et al are the broadest method to broadcast. They're not limited by geography, time of day, cable provider, and due to recording capabilities, the messages kept online easily without many restrictions. And potentially most importantly, they're easy for everyone to find. In 2018, a number of people sued Trump for him blocking them on Twitter. It was viewed as unconstitutional to be blocked from reading and engaging with his messages. That went to the Courts and they agreed, his Twitter account was declared a "public forum" and he had to unblock them. That's the real mismatch here.
- detaro 6y ago
  That has been an issue for a long time: ever since individual citizens became unable to use these platforms, or were expected to make compromises on privacy rights etc to do so.
- 6y ago
timonoko 6y ago
BTW. Has anybody ever seen ever one example of sequence of Tweets that are so cleverly constructed by some foreign entity that it might have effect on elections? Majority of (European) moderation in Twitter is done in Russia, or Russians residing in Malta or some such place. I used to burn Twitter accounts by the dozens by trolling^H^H^Hdiscussing issues that only Russians have opposing and sentimental view -- like Winter War, or origin of "Rus", or Crimea occupation.
- KboPAacDA3 6y ago
  Over the last 4 years I have personally tried finding an actual example, but still they allude me. I've seen plenty of bot activity, or news articles talking about bot capabilities, but never a real post in the wild or even a screenshot of one. I'm personally believe this is a mass hysteria used to explain how intelligent voters can make wrong decisions.
  - SCHiM 6y ago
    There's an example of how this looks in this teardown: https://gru.gq/2019/12/07/baltic-counter-disinformation-2/ https://gru.gq/2019/12/07/baltic-counter-disinformation-2/ This explains more background: https://gru.gq/2018/11/08/russian-propaganda-isnt-even-good/ https://gru.gq/2018/11/08/russian-propaganda-isnt-even-good/
  - kube-system 6y ago
    You might be interested in page 14-35 of the Mueller report. Yes, it's heavily redacted, but there is some very specific information in the unredacted portions, including individual twitter handles, etc. https://cdn.cnn.com/cnn/2019/images/04/18/mueller-report-searchable.pdf https://cdn.cnn.com/cnn/2019/images/04/18/mueller-report-sea...
- deleted 6y ago
  [deleted]
- tasty_freeze 6y ago
  Your challenge seems misplaced. It isn't that one or a few tweets or likes is clearly responsible for a significant shift of opinion (usually). Your question is like asking, "Has anybody seen the bucket of water which caused the flood?" In 2016 there were Russian accounts with hundreds of thousands of followers. They maintained accounts on both sides of contentious issues and pushed inflammatory stories simply to increase divisiveness. Stories they wanted to promote, whether they created it or not, they could use bots to push to higher visibility. It isn't as if any one vote made the difference.
Fricken 6y ago
"A medium gets the content it deserves." Said Marshall McLuhan. Twitter is a minimalist virtue signalling platform that selects for sensation and controversy. Nothing in the article should come as a surprise to anyone read up on their Chomsky. If Trump had used traditional presidential communication channels, his pulpit would not have have the same capacity for amplifying disinformation and fomenting outrage. The mainstream media earned many clicks in their expressions of outrage over the last several years. The whole media ecosystem was either complicit or counter-complicit to Trump's BS, and the national discourse is now a dumpster fire. It's not that the real issue is this or the real issue is that. I'd like to nominate "comorbidities" as the word of the year.
- uniqueid 6y ago
  It's not a Twitter problem specifically. It's an internet-wide problem. You can't hold users accountable if you can't even uniquely identify them. There's hardly any point trying to search for bots because they are, effectively, in infinite supply.
motohagiography 6y ago
Is there a significant difference between the manipulation activity on real-identity social media and pseudonymous media? FB and Twitter are places where the real participants mostly use their real identities, whereas places like Reddit and HN are just politely pseudonymous. I wonder if the collateral of real identities means a greater or lesser degree of manipulation.I don't use those other sites because I'm less interested in connecting online with people who use their real names, as it reduces to a personal brand performance divorced from ideas and truth. Whether we're engaged in discourse or politics seems linked to whether we're using "real" or abstracted identities. While I think people who can't abstract their ideas from their identities are the ones who get manipulated, consuming pseudonymously produced junk could have a similar but different impact.
ivanhoe 6y ago
Social media is just another media, and media hacve always been manipulated, somewhere more, somewhere less. This might be a news to Americans, as US propaganda efforts used to be more directed outwards, but us who grew up in (failed) socialism grew up knowing that most of political news are filtered lies and that you can't ever trust just one source. And I think that's the only viable solution, educating people to be skeptical of anything, on first ball at least. Check the multiple sources, wait a little for stories to be officially confirmed before reacting. It's not that hard, and Internet actually makes it a lot more viable, just that we still haven't really adapted emotionally to how the world works now. We still jump to conclusions, we still click the clickbait titles and the biggest problem, we're all deeply biased on our own views of the world - and these biases are the glasses that we view the world through, they affect every single aspect of our perception, often on a subconscious level. People often believe in fake news simply because deep inside they already think that's the case, they want it to be true.
- mistermann 6y ago
  There are these two young fish swimming along and they happen to meet an older fish swimming the other way, who nods at them and says “Morning, boys. How’s the water?” And the two young fish swim on for a bit, and then eventually one of them looks over at the other and goes “What the hell is water?” How much of the accepted-without-question, 100% undisputed mainstream narrative is disinformation as well? Just two recent examples: 1. Voter fraud in the election. After the election was done and rumours of fraud arose in the social media memeplex, nearly immediately the "There is no evidence of election fraud - claims of election fraud are disinformation" counter-meme was released into official media channels (and from there spread into social media). That this too is disinformation should be obvious. - Election fraud (to whatever degree) occurred, or it did not. - Evidence of election fraud (that is discoverable by humans) then existed, or it did not (regardless of that discovery). - The necessary investigations to discover any such evidence had been launched and completed, or they had not. disinformation: false information which is intended to mislead, especially propaganda issued by a government organization to a rival power or the media. As of the date of the election completion + 1 day, the necessary investigations to uncover the existence of election fraud had not been completed - therefore, the assertions of fact broadcast from multiple independent media organizations that there is(!) no evidence of election fraud is disinformation. 2. The riot at the Capitol Immediately after the Capitol incident, a coincidentally similar narrative was asserted by multiple independent media organizations: this was literally a coup attempt, an attack on "our most sacred(!) institution". And, the intent of the people on the ground was(!) to execute a coup against the US government. This was asserted as if it was a fact. However, what was actually going on (which is contained within the minds of the human beings who were there that day) is not actually known. (Meanwhile, speaking of "attacks on sacred institutions": https://en.wikipedia.org/wiki/United_States_involvement_in_regime_change https://en.wikipedia.org/wiki/United_States_involvement_in_r... - curiously, this seemed to not come up in the conversation, despite its obvious relevance). Getting back to the "Morning, boys. How’s the water?" story...somewhere along the way (my sense is that this has occurred mostly in the last 5 years), the notion of what is actually(!) true seems to have become somehow radioactive in a way - the very idea is considered repulsive, not to be mentioned in polite company. And it's not just braindead political partisans who exhibit this behavior - as far as I can tell, this default intuition has been adopted by extremely high percentages of people across all categorizations. If anything, it seems to me that highly intelligent people are often even more repulsed by it than normal folks (who perhaps don't grasp the philosophical depth of the idea). The election fraud disagreement and the Capitol riot, these things concern me somewhat, but what concerns me even more is the degradation of the very fabric of reality, the "This is Water" in the story. And not only the degradation itself, but that the degradation is being promoted (at least unconsciously and passively) by some of the brightest minds in our population. A society that is arguably involved in some form of a new cold war with a rapidly rising new global superpower adopting a culture of disregard for what is actually true seems like not a great idea to me. If we actually care as much about the Capitol riots as we proclaim to, I think we should be willing to face up to what the actual underlying causes of it are, as opposed to the simplistic narratives (disinformation) that are asserted as fact to be the cause.
justinzollars 6y ago
Perhaps Twitter should label bots?
deleted 6y ago
[deleted]
throwawaytolk 6y ago
propaganda is the oldest trick in the book, goes all the way back to the first civilizations. I once found this stellar talk on YouTube about the history of the NSA up from the cabinet noir days in ww1. What was particularly interesting to me is that you can look at ww1 propaganda efforts and show hard numbers that they're extremely effective at changing viewpoints.
2OEH8eoCRo0 6y ago
I've suspected this for years- twitter and reddit are especially manipulated. You don't even need an email for reddit and it's mostly browsed by Americans. You'd have to have your head in the sand to not think it's manipulated like crazy. They're huge juicy targets with incredibly low barriers. Anytime I'd try to bring the conversation back toward civility I'd be downvoted to oblivion. I find it's best to separate news from social.
cudgy 6y ago
The article is missing some very important points. How effective are these disinformation campaigns? Are they effective enough to, for example, significantly impact an election or sway enough people’s opinions about a vaccine to affect it’s adoption? Where is the evidence? Also, how is a bot able to avoid detection? Seems like it would be easy to see that a single userbot “liked” more than is humanly possible ... even if the bot were written to mimic normal humans, then it would require more bots, all presumably coming from a similar cluster of ip addresses, making it more easy to detect. Of course, what incentive do the social media platforms have to detect fake users (especially active ones like bots), since their metrics are largely driven by that metric?

162 points by docdeek 6y ago

api 6y ago

That is an issue, but the real fundamental issue is algorithmic timelines and recommendations that bias content to maximize engagement. What maximizes engagement? Hate, fear, and bombast.

im3w1l 6y ago
https://en.wikipedia.org/wiki/List_of_most-liked_tweets https://en.wikipedia.org/wiki/List_of_most-liked_tweets https://en.wikipedia.org/wiki/List_of_most-retweeted_tweets https://en.wikipedia.org/wiki/List_of_most-retweeted_tweets Doesn't seem like you are right about that. Edit: Going by replies might yield a different more negative result, the word "ratiod" imply that many see a reply almost as a downvote. But I was not able to find a list of most replied-to tweets.
- true_religion 6y ago
  According to that list, K-pop maximizes engagement so Twitter should pivot to being MTV online.
- hshshs2 6y ago
  replies are only similar to downvotes in the absence of likes, that’s why it’s “ratiod” and not “replied”
- ben_w 6y ago
  Interesting data — and heartwarming to see! — but I’m not sure it’s the right measure. A thousand tweets that are each seen by a different (non-intersecting) group of a thousand people, each tweet being an accusation of evil directed at a specific person or group, does as much damage as one tweet seen by a million. I hope that this measure would show the same result as the two lists you linked to; my current guess is that if someone did this analysis, the result would be almost as positive, but not quite, on the grounds that hate-inducing tweets are against the TOS and will be hidden or deleted when they reach a visibility threshold.
  - AnimalMuppet 6y ago
    A thousand tweets, each seen by a different, overlapping group of a thousand people, reaches fewer total people, but to those people it looks like multiple independent sources confirming the same information. That is, the information looks more trustworthy. (That was one of the points of the Soviet "active measures" propaganda during the Cold War. They not only had western sources giving the Soviet line, they had multiple sources giving the same line.)
    - ben_w 6y ago
      Thanks for the insight, I thought that might be the case but didn’t know. (I also felt it would be an Apples-to-Oranges comparison when the alternative was a single account tweeting to a million followers).
- dahfizz 6y ago
  I don't think looking at the 30 most liked / retweeted tweets is a good measure for deciding what kind of content is most engaging. BTS makes up more than half of both of those lists - that's just a list of popular twitter accounts. The most tweeted about people on twitter are Trump, Biden, and George Floyd[1]. I think that gives a pretty good idea of the content occupying people's minds. [1] https://blog.twitter.com/en_us/topics/insights/2020/spending-2020-together-on-twitter.html https://blog.twitter.com/en_us/topics/insights/2020/spending...
w_t_payne 6y ago
This is the fundamental design flaw of the system. Fixing it isn't going to be easy, though. Look at the amount of effort that the scientific community tries to put into improving the quality of communication:- peer review, standards for content ... and there's still nagging issues with reproducing results and even outright fraud in some cases. Whatever solution we come up with will likely increase friction in the user experience, but there's lots of room for innovation here.
- api 6y ago
  Fixing this isn't a matter of adding complexity but of removing it. When timelines were sequential or user-moderated and YouTube was just a search engine and host for videos it was fine. Algorithmic prioritization algorithms were introduced around the 2008-2012 time frame and all Internet discourse went to hell shortly afterwords. What we did was to insert a biasing function between all people that biased content for engagement. What maximizes engagement? Hate, fear, and conflict.
- AnimalMuppet 6y ago
  The fundamental flaw is human nature. That's not going to be easy to fix.

Dem_Boys 6y ago

Since the first day I meddled in creating online bots I've had the intuition that large scale manipulation was occurring. If a single developer can create a twitter bot in 4 hours that likes posts about a topic, then what could a large company, political party, or entire country do? I've always thought "Well they're just manipulating online opinions" but the capital and blm riots have shown that online opinions can have real world impacts. Family and friends often are distressed by the state of the world, where their view of the world is social media. But how much of social media is real? How many of the radical opinions expressed are made by real people? Of those real people, how many of them were influenced by bots or "computational propaganda"? I believe that we, even really smart engineers, are easier to influence that we want to believe

octocop 6y ago
> Family and friends often are distressed by the state of the world I see this a lot too, and I have seen it the other way around as well. Where people believe everything is either Armageddon or a non-issue. Ironically I think this effect is applied to itself. Are we distressed because we think our world-view is wrong so we seek to the extremes for validation?
- Dem_Boys 6y ago
  I'm no expert but this is my opinion. 1. We are all products of our environment 2. The internet and social media is becoming a higher percentage of our environment every day 3. Large parts of the internet and social media makes money by occupying our attention 4. The best way(that we know of) to occupy our attention for long periods is through hyperbolic/enraging material ^^^ Doing this long enough leads to enraged/hyperbolic population and I don't know of a reasonable way to stop it :(
  - shaolinspirit 6y ago
    i would disagree with #1, we are products of our curation and observation of information that is coming in forms of feelings or outer causes. you can be in a bad environment but still curate where do you put your attention by explicitly acknowledging what subjective values do you carry during your lifetime. as above, curation is the main skill of the future, you can use youtube to watch stupid cat videos or to watch self development content, it is up to the viewer. if you are willing to read lies, you deserve to be deceived. with the current abundance of information and possibilities it is a blasphemy to argue that people are manipulated, they are rather choosing this by being lazy or not curious enough or because it already confirms their biases / beliefs. it is sad to see that people replicated the TV (Tell-Me-Vision) model on the internet.

mnw21cam 6y ago

Surely we can widen the issue from state-backed manipulation to lots-of-money-backed manipulation.

BrandonMarc 6y ago
... and the social media giants have lots of money, and are in the perfect position to manipulate. Motivation and opportunity ...
Miner49er 6y ago
And the article's suggested solution is: > These companies should therefore increase their efforts to flag and remove disinformation, along with all cyber troop accounts which are used to spread harmful content online. So the suggested solution is to have different lots-of-money companies stop the lots-of-money manipulation. Of course, if you are doing the right kind of manipulation (ads), that's allowed. Seems a bit hypocritical to me.
mindslight 6y ago
Lots-of-money-backed manipulation is how things have been traditionally. "Never pick a fight with someone who buys ink by the barrel". It tends to be a stabilizing force for society, despite the malevolence (eg Iraq war). Meanwhile state-backed (or alternatively, lots-of-foreign-money-backed) seems to be a destabilizing force. We've presently got two large political movements, both essentially protesting against out of control government, but yet they've ended up being each other's enemy.
- mnw21cam 6y ago
  I don't think the distinction between state-backed or lots-of-money-backed manipulation is really what makes it stabilising or destabilising. Rather, I think it whether the manipulation separates out the manipulated into an echo-chamber of similarly-minded outrage or not. For example, a government statement based on scientific investigation, such as "Covid-19 is a dangerous disease, but it helps if you wear a mask" is state-backed manipulation, but it doesn't try to isolate the manipulated into an echo-chamber, and it is intending to be stabilising. Conversely, some of the worst parts of social media can whip up a storm of outrage and hatred because the algorithm (which is intended to maximise the advertising profits of a commercial company) recommends outrageous stories that make you angry, but that you are likely to agree with. This is lots-of-money-backed manipulation, but it does isolate people into echo-chambers, and it is destabilising. Regarding having two large political movements that have become enemies, I think this is an emergent property of a complex interaction between politics and the tendency of people to get sucked into echo-chambers that (as an emergent property, not an intended outcome) steer people towards thinking that the other side is the devil incarnate. Political parties will always disagree with each other. I admire politicians who can say to each other that they disagree strongly with the other's point of view, but they respect them as a person and get on with them well. I think it will be very hard to destroy the natural tendency of social media to form echo-chambers, but it should be attempted. Regarding politics, I think it's interesting that countries that have election systems that discourage two-party politics in favour of multiple parties having to make compromises with each other tend to be more stable and friendly about it.

VBprogrammer 6y ago

To me the biggest issue that was raised by the Trump bans is that we shouldn't be relying on these channels for communication with our governments and elected representatives. Boris Johnson had a brief flirtation with Facebook Q&A sessions. Our governments should broadcast any messaging in the widest possible sense. Official government websites, live feeds to anyone who cares for it etc.

krapp 6y ago
The only reason this was ever an issue is that Donald Trump chose to use his personal Twitter account as his primary means of communication, to the point that it had to be recognized as a de facto official channel. And that's mostly because he believed the press and every other avenue was controlled by leftists. This is far more a "Donald Trump is a paranoiac" problem than a social media problem per se. Nothing wrong with a politican doing an AMA on Reddit or a Facebook Q&A or having a Twitter account, which they all do at this point. Politicians using social media as a convenient way to communicate with people directly can be a net benefit for society, although probably more so the more local government becomes. But using social media exclusive of other, official channels is obviously a problem.
caseysoftware 6y ago
Twitter, Facebook, YouTube, et al are the broadest method to broadcast. They're not limited by geography, time of day, cable provider, and due to recording capabilities, the messages kept online easily without many restrictions. And potentially most importantly, they're easy for everyone to find. In 2018, a number of people sued Trump for him blocking them on Twitter. It was viewed as unconstitutional to be blocked from reading and engaging with his messages. That went to the Courts and they agreed, his Twitter account was declared a "public forum" and he had to unblock them. That's the real mismatch here.
detaro 6y ago
That has been an issue for a long time: ever since individual citizens became unable to use these platforms, or were expected to make compromises on privacy rights etc to do so.
6y ago

timonoko 6y ago

BTW. Has anybody ever seen ever one example of sequence of Tweets that are so cleverly constructed by some foreign entity that it might have effect on elections? Majority of (European) moderation in Twitter is done in Russia, or Russians residing in Malta or some such place. I used to burn Twitter accounts by the dozens by trolling^H^H^Hdiscussing issues that only Russians have opposing and sentimental view -- like Winter War, or origin of "Rus", or Crimea occupation.

KboPAacDA3 6y ago
Over the last 4 years I have personally tried finding an actual example, but still they allude me. I've seen plenty of bot activity, or news articles talking about bot capabilities, but never a real post in the wild or even a screenshot of one. I'm personally believe this is a mass hysteria used to explain how intelligent voters can make wrong decisions.
- SCHiM 6y ago
  There's an example of how this looks in this teardown: https://gru.gq/2019/12/07/baltic-counter-disinformation-2/ https://gru.gq/2019/12/07/baltic-counter-disinformation-2/ This explains more background: https://gru.gq/2018/11/08/russian-propaganda-isnt-even-good/ https://gru.gq/2018/11/08/russian-propaganda-isnt-even-good/
- kube-system 6y ago
  You might be interested in page 14-35 of the Mueller report. Yes, it's heavily redacted, but there is some very specific information in the unredacted portions, including individual twitter handles, etc. https://cdn.cnn.com/cnn/2019/images/04/18/mueller-report-searchable.pdf https://cdn.cnn.com/cnn/2019/images/04/18/mueller-report-sea...
deleted 6y ago
[deleted]
tasty_freeze 6y ago
Your challenge seems misplaced. It isn't that one or a few tweets or likes is clearly responsible for a significant shift of opinion (usually). Your question is like asking, "Has anybody seen the bucket of water which caused the flood?" In 2016 there were Russian accounts with hundreds of thousands of followers. They maintained accounts on both sides of contentious issues and pushed inflammatory stories simply to increase divisiveness. Stories they wanted to promote, whether they created it or not, they could use bots to push to higher visibility. It isn't as if any one vote made the difference.

Fricken 6y ago

"A medium gets the content it deserves." Said Marshall McLuhan. Twitter is a minimalist virtue signalling platform that selects for sensation and controversy. Nothing in the article should come as a surprise to anyone read up on their Chomsky. If Trump had used traditional presidential communication channels, his pulpit would not have have the same capacity for amplifying disinformation and fomenting outrage. The mainstream media earned many clicks in their expressions of outrage over the last several years. The whole media ecosystem was either complicit or counter-complicit to Trump's BS, and the national discourse is now a dumpster fire. It's not that the real issue is this or the real issue is that. I'd like to nominate "comorbidities" as the word of the year.

uniqueid 6y ago
It's not a Twitter problem specifically. It's an internet-wide problem. You can't hold users accountable if you can't even uniquely identify them. There's hardly any point trying to search for bots because they are, effectively, in infinite supply.

motohagiography 6y ago

Is there a significant difference between the manipulation activity on real-identity social media and pseudonymous media? FB and Twitter are places where the real participants mostly use their real identities, whereas places like Reddit and HN are just politely pseudonymous. I wonder if the collateral of real identities means a greater or lesser degree of manipulation.I don't use those other sites because I'm less interested in connecting online with people who use their real names, as it reduces to a personal brand performance divorced from ideas and truth. Whether we're engaged in discourse or politics seems linked to whether we're using "real" or abstracted identities. While I think people who can't abstract their ideas from their identities are the ones who get manipulated, consuming pseudonymously produced junk could have a similar but different impact.

ivanhoe 6y ago

Social media is just another media, and media hacve always been manipulated, somewhere more, somewhere less. This might be a news to Americans, as US propaganda efforts used to be more directed outwards, but us who grew up in (failed) socialism grew up knowing that most of political news are filtered lies and that you can't ever trust just one source. And I think that's the only viable solution, educating people to be skeptical of anything, on first ball at least. Check the multiple sources, wait a little for stories to be officially confirmed before reacting. It's not that hard, and Internet actually makes it a lot more viable, just that we still haven't really adapted emotionally to how the world works now. We still jump to conclusions, we still click the clickbait titles and the biggest problem, we're all deeply biased on our own views of the world - and these biases are the glasses that we view the world through, they affect every single aspect of our perception, often on a subconscious level. People often believe in fake news simply because deep inside they already think that's the case, they want it to be true.

mistermann 6y ago
There are these two young fish swimming along and they happen to meet an older fish swimming the other way, who nods at them and says “Morning, boys. How’s the water?” And the two young fish swim on for a bit, and then eventually one of them looks over at the other and goes “What the hell is water?” How much of the accepted-without-question, 100% undisputed mainstream narrative is disinformation as well? Just two recent examples: 1. Voter fraud in the election. After the election was done and rumours of fraud arose in the social media memeplex, nearly immediately the "There is no evidence of election fraud - claims of election fraud are disinformation" counter-meme was released into official media channels (and from there spread into social media). That this too is disinformation should be obvious. - Election fraud (to whatever degree) occurred, or it did not. - Evidence of election fraud (that is discoverable by humans) then existed, or it did not (regardless of that discovery). - The necessary investigations to discover any such evidence had been launched and completed, or they had not. disinformation: false information which is intended to mislead, especially propaganda issued by a government organization to a rival power or the media. As of the date of the election completion + 1 day, the necessary investigations to uncover the existence of election fraud had not been completed - therefore, the assertions of fact broadcast from multiple independent media organizations that there is(!) no evidence of election fraud is disinformation. 2. The riot at the Capitol Immediately after the Capitol incident, a coincidentally similar narrative was asserted by multiple independent media organizations: this was literally a coup attempt, an attack on "our most sacred(!) institution". And, the intent of the people on the ground was(!) to execute a coup against the US government. This was asserted as if it was a fact. However, what was actually going on (which is contained within the minds of the human beings who were there that day) is not actually known. (Meanwhile, speaking of "attacks on sacred institutions": https://en.wikipedia.org/wiki/United_States_involvement_in_regime_change https://en.wikipedia.org/wiki/United_States_involvement_in_r... - curiously, this seemed to not come up in the conversation, despite its obvious relevance). Getting back to the "Morning, boys. How’s the water?" story...somewhere along the way (my sense is that this has occurred mostly in the last 5 years), the notion of what is actually(!) true seems to have become somehow radioactive in a way - the very idea is considered repulsive, not to be mentioned in polite company. And it's not just braindead political partisans who exhibit this behavior - as far as I can tell, this default intuition has been adopted by extremely high percentages of people across all categorizations. If anything, it seems to me that highly intelligent people are often even more repulsed by it than normal folks (who perhaps don't grasp the philosophical depth of the idea). The election fraud disagreement and the Capitol riot, these things concern me somewhat, but what concerns me even more is the degradation of the very fabric of reality, the "This is Water" in the story. And not only the degradation itself, but that the degradation is being promoted (at least unconsciously and passively) by some of the brightest minds in our population. A society that is arguably involved in some form of a new cold war with a rapidly rising new global superpower adopting a culture of disregard for what is actually true seems like not a great idea to me. If we actually care as much about the Capitol riots as we proclaim to, I think we should be willing to face up to what the actual underlying causes of it are, as opposed to the simplistic narratives (disinformation) that are asserted as fact to be the cause.

justinzollars 6y ago

Perhaps Twitter should label bots?

deleted 6y ago

[deleted]

throwawaytolk 6y ago

propaganda is the oldest trick in the book, goes all the way back to the first civilizations. I once found this stellar talk on YouTube about the history of the NSA up from the cabinet noir days in ww1. What was particularly interesting to me is that you can look at ww1 propaganda efforts and show hard numbers that they're extremely effective at changing viewpoints.

2OEH8eoCRo0 6y ago

I've suspected this for years- twitter and reddit are especially manipulated. You don't even need an email for reddit and it's mostly browsed by Americans. You'd have to have your head in the sand to not think it's manipulated like crazy. They're huge juicy targets with incredibly low barriers. Anytime I'd try to bring the conversation back toward civility I'd be downvoted to oblivion. I find it's best to separate news from social.

cudgy 6y ago

The article is missing some very important points. How effective are these disinformation campaigns? Are they effective enough to, for example, significantly impact an election or sway enough people’s opinions about a vaccine to affect it’s adoption? Where is the evidence? Also, how is a bot able to avoid detection? Seems like it would be easy to see that a single userbot “liked” more than is humanly possible ... even if the bot were written to mimic normal humans, then it would require more bots, all presumably coming from a similar cluster of ip addresses, making it more easy to detect. Of course, what incentive do the social media platforms have to detect fake users (especially active ones like bots), since their metrics are largely driven by that metric?