8 ms·

Note that nothing in the article is AI-specific: the entire argument is built around the cost of persuasion, with the potential of AI to more cheaply generate p

• smartmic 11mo ago
  But AI is next in line as a tool to accelerate this, and it has an even greater impact than social media or troll armies. I think one lever is working towards "enforced conformity." I wrote about some of my thoughts in a blog article[0]. [0]: https://smartmic.bearblog.dev/enforced-conformity/ https://smartmic.bearblog.dev/enforced-conformity/
  • citrin_ru 11mo ago
    But social networks is the reason one needs (benefits from) trolls and AI. If you own a traditional media outlet you need somehow to convince people to read/watch it. Ads can help but it’s expensive. LLM can help with creating fake videos but computer graphics was already used for this. With modern algorithmic social networks you instead can game the feed and even people who would not choose you media will start to see your posts. End even posts they want to see can be flooded with comment trying to convince in whatever is paid for. It’s cheaper than political advertising and not bound by the law. Before AI it was done by trolls on payroll and now they can either maintain 10x more fake accounts or completely automate fake accounts using AI agents.
    • andsoitis 11mo ago
      Social networks are not a prerequisite for sentiment shaping by AI. Every time you interact with an AI, its responses and persuasive capabilities shape how you think.
  • andy99 11mo ago
    See also https://english.elpais.com/society/2025-03-23/why-everything-looks-the-same-everywhere.html https://english.elpais.com/society/2025-03-23/why-everything... https://medium.com/knowable/why-everything-looks-the-same-bad80133dd6e https://medium.com/knowable/why-everything-looks-the-same-ba...
  • themafia 11mo ago
    People are naturally conform _themselves_ to social expectations. You don't need to enforce anything. If you alter their perception of those expectations you can manipulate them into taking actions under false pretenses. It's a abstract form of lying. It's astroturfing at a "hyperscale." The problem is this only seems to work best when the technique is used sparingly and the messages are delivered through multiple media avenues simultaneously. I think there's very weak returns particularly when multiple actors use the techniques at the same time in opposition to each other and limited to social media. Once people perceive a social stale mate they either avoid the issue or use their personal experiences to make their decisions.
    • pixl97 11mo ago
      >Once people perceive a social stale mate they either avoid This is called the Firehose of Falsehood and it's a very effective way of killing public participation. >use their personal experiences to make their decisions Yes they can if they have them. But people use other peoples personal experiences when they don't, which means all you have to do is become their facebook friend and then tell them that 'trans mexican aliens from mars stole their job' and they'll start repeating it as a personal experience.
• go_elmo 11mo ago
  Good point - its not a previously inexistent mechanism - but AI leverages it even more. A russian troll can put out 10x more content with automation. Genuine counter-movements (e.g. grassroot preferences) might not be as leveraged, causing the system to be more heavily influenced by the clearly pursued goals (which are often malicious)
  • andsoitis 11mo ago
    > Genuine counter-movements (e.g. grassroot preferences) might not be as leveraged Then that doesn’t seem like a (counter) movement. There are also many “grass roots movements” that I don’t like and it doesn’t make them “good” just because they’re “grass roots”.
    • none2585 11mo ago
      In this context grass roots would imply the interests of a group of common people in a democracy (as opposed to the interests of a small group of elites) which ostensibly is the point.
      • andsoitis 11mo ago
        I think it is more useful to think of “common people” and “the elites” not as separate categories but rather than phases on a spectrum, especially when you consider very specific interests. I have some shared interested with “the common people” and some with “the elites”.
      • Nasrudith 11mo ago
        "The interests of the common people" is a very dangerous collection of weasel words. The whole damn point of democracy is realizing that you cannot just delegate power to 'better people' and expect them to rule in interest of the common people. Because that is basically exactly what royalists were promising!
  • mdotmertens 11mo ago
    It's not only about efficiency. When AI is utilized, things can become more personal and even more persuasive. If AI psychosis exists, it can be easy for untrained minds to succumb to these schemes.
• xbmcuser 11mo ago
  [flagged]
• citrin_ru 11mo ago
  AI (LLM) is a force multiplier for troll armies. For the same money bad actors can brainwash more people.
  • yorwba 11mo ago
    Alternatively, since brainwashing is a fiction trope that doesn't work in the real world, they can brainwash the same (0) number of people for less money. Or, more realistically, companies selling social media influence operations as a service will increase their profit margins by charging the same for less work.
    • djmips 11mo ago
      So your thesis is that marketing doesn't work?
      • yorwba 11mo ago
        My thesis is that marketing doesn't brainwash people. You can use marketing to increase awareness of your product, which in turn increases sales when people would e.g. otherwise have bought from a competitor, but you can't magically make arbitrary people buy an arbitrary product using the power of marketing.
        • FridayoLeary 11mo ago
          This. I believe people massively exaggerate the influence of social engineering as a form of coping. "they only voted for x because they are dumb and blindly fell for russian misinformation." reality is more nuanced. It's true that marketers for the last century have figured out social engineering but it's not some kind of magic persuasion tool. People still have free will and choice and some ability to discern truth from falsehood.
        • Barrin92 11mo ago
          so you just object to the semantics of 'brainwashing'? No influence operation needs to convince an arbitrary amount of people of arbitrary products. In the US nudging a few hundred thousand people 10% in one direction wins you an election.
• zaptheimpaler 11mo ago
  Making something 2x cheaper is just a difference in quantity, but 100x cheaper and easier becomes a difference in kind as well.
  • HPsquared 11mo ago
    "Quantity has a quality of its own."
    • deleted 11mo ago
      [deleted]
• SCdF 11mo ago
  I've only read the abstract, but there is also plenty of evidence to suggest that people trust the output of LLMs more than other forms of media (or that they should). Partially because it feels like it comes from a place of authority, and partially because of how self confident AI always sounds. The LLM bot army stuff is concerning, sure. The real concern for me is incredibly rich people with no empathy for you or I, having interstitial control of that kind of messaging. See, all of the grok ai tweaks over the past however long.
  • prox 11mo ago
    And just see all of history where totalitarians or despotic kings were in power.
  • andsoitis 11mo ago
    Do you think these super wealthy people who control AI use the AI themselves? Do you think they are also “manipulated” by their own tool or do they, somehow, escape that capture?
    • pjc50 11mo ago
      It's fairly clear from Twitter that it's possible to be a victim of your own system. But sycophancy has always been a problem for elites. It's very easy to surround yourselves with people who always say yes, and now you can have a machine do it too. This is how you get things like the colossal Facebook writeoff of "metaverse".
    • wongarsu 11mo ago
      Isn't Grok just built as "the AI Elon Musk wants to use"? Starting from the goals of being "maximally truth seeking" and having no "woke" alignment and fewer safety rails, to the various "tweaks" to the Grok Twitter bot that happen to be related to Musk's world view Even Grok at one point looking up how Musk feels about a topic before answering fits that pattern. Not something that's healthy or that he would likely prefer when asked, but something that would produce answers that he personally likes when using it
• t_mann 11mo ago
  Sounds like saying that nothing about the Industrial Revolution was steam-machine-specific. Cost changes can still represent fundamental shifts in terms of what's possible, "cost" here is just an economists' way of saying technology.
• muldvarp 11mo ago
  But the entire promise of AI is that things that were expensive because they required human labor are now cheap. So if good things happening more because AI made them cheap is an advantage of AI, then bad things happening more because AI made them cheap is a disasvantage of AI.
• pbreit 11mo ago
  Considering that LLMs have substantially "better" opinions than, say, the MSM or social media, is this actually a good thing? Might we avoid the whole woke or pro-Hamas debacles? Maybe we could even move past the current "elites are intrinsically bad" era?
  • crashmat 11mo ago
    You appear to be exactly the kind of person the article is talking about. What exactly makes LLMs have "better" opinions than others?
  • windexh8er 11mo ago
    LLMs don't have "opinions" [0] because they don't actually think. Maybe we need to move past the ignorance surrounding how LLMs actually work, first. [0] https://www.theverge.com/ai-artificial-intelligence/827820/large-language-models-ai-intelligence-neuroscience-problems https://www.theverge.com/ai-artificial-intelligence/827820/l...
  • throwawayqqq11 11mo ago
    Yes please, lets move past the "elites bad" mind virus and finally turn our attention to solutions for the many, like cost of living and deteriorating public services. We have to trust the elites! Migrants are the actual root cause! I hope you still have traces of critical thinking left to spot the sarcasm.
• justsomejew 11mo ago
  [flagged]
  • Arainach 11mo ago
    Russian mass influence campaigns are well documented globally and have been for more than a decade.
    • justsomejew 11mo ago
      Of course, of course.. still, strangely I see online other kinds of "armies" much more often.. and the scale, in this case, is indeed of armies..
      • OKRainbowKid 11mo ago
        Whataboutism, to me, seems like one of the most important tools of the Russian troll army.
        • justsomejew 11mo ago
          Well, counting the number of "non trolls" here, and my own three comments, surely shows the Russian hords in action ;)
    • Libidinalecon 11mo ago
      It is also right in their military strategy text that you can read yourself. Even beyond that, why would an adversarial nation state to the US not do this? It is extremely asymmetrical, effective and cheap. The parent comment shows how easy it is to manipulate smart people away from their common sense into believing obvious nonsense if you use your brain for 2 seconds.
  • avhception 11mo ago
    Are you implying that the "neo-KGB" never mounted a concerted effort to manipulate western public opinion through comment spam? We can debate whether that should be called a "troll army", but we're fairly certain that such efforts are made, no?
• gaigalas 11mo ago
  > nothing in the article is AI-specific Timing is. Before AI this was generally seen as crackpot talk. Now it is much more believable.
  • lazide 11mo ago
    It’s been pretty transparently happening for years in most online communities.
    • pixl97 11mo ago
      And yet denied massively. This said, if I were one of the people performing mass manipulation, I would also make bots to say mass manipulation is not really.
  • vladms 11mo ago
    You mean the failed persuasions were "crackpot talk" and the successful ones were "status quo". For example, a lot of persuasion was historically done via religion (seemingly not mentioned at all in the article!) with sects beginning as "crackpot talk" until they could stand on their own.
    • gaigalas 11mo ago
      What I mean is that talking about mass persuation was (and to a certain degree, it still is) crackpot talk. I'm not talking about the persuations themselves, it's the general public perception of someone or some group that raises awareness about it. This also excludes ludic talk about it (people who just generally enjoy post-apocalyptic aesthetics but doesn't actually consider it to be a thing that can happen). 5 years ago, if you brought up serious talk about mass systemic persuation, you were either a lunatic or a philosopher, or both.
  • wongarsu 11mo ago
    Social media has been flooded by paid actors and bots for about a decade. Arguably ever since Occupy Wall Street and the Arab Spring showed how powerful social media and grassroots movements could be, but with a very visible and measurable increase in 2016
• bjourne 11mo ago
  While true in principle, you are underestimating the potential of ai to sway people's opinions. "@grok is this true" is already a meme on Twitter and it is only going to get worse. People are susceptible to eloquent bs generated by bots.
  • deleted 11mo ago
    [deleted]
• ekjhgkejhgk 11mo ago
  > Note that nothing in the article is AI-specific No one is arguing that the concept of persuasion didn't exist before AI. The point is that AI lowers the cost. Yes, Russian troll armies also have a lower cost compared to going door to door talking to people. And AI has a cost that is lower still.
• tgv 11mo ago
  That's one of those "nothing to see here, move along" comments. First, generative AI already changed social dynamics, in spite of facebook and all that being around for more than a decade. People trust AI output, much more than a facebook ad. It can slip its convictions into every reply it makes. Second, control over the output of AI models is limited to a very select few. That's rather different from access to facebook. The combination of those two factors does warrant the title.
• kev009 11mo ago
  Yup "could shape".. I mean this has been going on time immemorial. It was odd to see random nerds who hated Bill Gates the software despot morph into acksually he does a lot of good philanthropy in my lifetime but the floodgates are wide open for all kinds of bizarre public behavior from oligarchs these days. The game is old as well as evergreen. Hearst, Nobel, Howard Huges come to mind of old. Musk with Twitter, Ellison with TikTok, Bezos with Washington Post these days etc. The costs are already insignificant because they generally control other people's money to run these things.
  • UpsideDownRide 11mo ago
    Your example is weird tbh. Gates was doing capitalist things that were evil. His philanthropy is good. There is no contradiction here. People can do good and bad things.
    • kev009 11mo ago
      The "philanthropy" worked on you.
      • throwawayqqq11 11mo ago
        His differenciated oppinion did not on you.
• sam-cop-vimes 11mo ago
  Well, AI has certainly made it easier to make tailored propaganda. If an AI is given instructions about what messaging to spread, it can map out a path from where it perceives the user to where its overlords want them to be. Given how effective LLMs are at using language, and given that AI companies are able to tweak its behaviour, this is a clear and present danger, much more so than facebook ads.
• jacquesm 11mo ago
  That's a pretty typical middle-brow dismissal but it entirely misses the point of TFA: you don't need AI for this, but AI makes it so much cheaper to do this that it becomes a qualitative change rather than a quantitative one. Compared to that 'russian troll army' you can do this by your lonesome spending a tiny fraction of what that troll army would cost you and it would require zero effort in organization compared to that. This is a real problem and for you to dismiss it out of hand is a bit of a short-cut.
• rsynnott 11mo ago
  Making doing bad things way cheaper _is_ a problem, though.
• odiroot 11mo ago
  It has been practiced by populist politicians for millennia, e.g. pork barelling.
• ddlsmurf 11mo ago
  What makes AI a unique new threat is that it do a new kind of both surgical and mass attack: you can now generate the ideal message per target, basically you can whisper to everyone, or each group, at any granularity, the most convincing message. It also removes a lot of language and culture barriers, for ex. Russian or Chinese propaganda is ridiculously bad when it crosses borders, at least when targeting the english speaking world, this is also a lot easier/cheaper.
• tim333 11mo ago
  Also I think AI at least in its current LLM form may be a force against polarisation. Like if you go on X/twitter and type "Biden" or "Biden Crooked" in the "Explore" thing in the side menu you get loads of abusive stuff including the president slagging him off. Type into "Grok" about those it says Biden was a decent bloke and more "there is no conclusive evidence that Joe Biden personally committed criminal acts, accepted bribes, or abused his office for family gain" I mention Grok because being owned by a right leaning billionaire you'd think it'd be one of the first to go.
• coldtea 11mo ago
  >Note that nothing in the article is AI-specific: the entire argument is built around the cost of persuasion, with the potential of AI to more cheaply generate propaganda as buzzword link. That's the entire point, that AI cheapens the cost of persuassion. A bad thing X vs a bad thing X with a force multiplier/accelerator that makes it 1000x as easy, cheap, and fast to perform is hardly the same thing. AI is the force multiplier in this case. That we could of course also do persuassion pre-AI is irrelevant, same way when we talk about the industrial revolution the fact that a craftsman could manually make the same products without machines is irrelevant as to the impact of the industrial revolution, and its standing as a standalone historical era.
• dfxm12 11mo ago
  It is worth pointing out that ownership of AI is becoming more and more consolidated over time, by elites. Only Elon Musk or Sam Altman can adjust their AI models. We recognize the consolidation of media outlets as a problem for similar reasons, and Musk owning grok and twitter is especially dangerous in this regard. Conversely, buying facebook ads is more democratized.
  • thunderfork 11mo ago
    [dead]
• insane_dreamer 11mo ago
  > You don't need any AI for this. AI accelerates it considerably and with it being pushed everywhere, weaves it into the fabric of most of what you interact with. If instead of searches you now have AI queries, then everyone gets the same narrative, created by the LLM (or a few different narratives from the few models out there). And the vast majority of people won't know it. If LLMs become the de-facto source of information by virtue of their ubiquity, then voila, you now have a few large corporations who control the source of information for the vast majority of the population. And unlike cable TV news which I have to go out of my way to sign up and pay for, LLMs are/will be everywhere and available for free (ad-based). We already know models can be tuned to have biases (see Grok).
• zahlman 11mo ago
  The thread started with your reasonable observation but degenerated into the usual red-vs-blue slapfight powered by the exact "elite shaping of mass preferences" and "cheaply generated propaganda" at issue. > Comments should get more thoughtful and substantive, not less, as a topic gets more divisive. I'm disappointed.
  • notabee 11mo ago
    I'm pretty much always disappointed these days reading online discussions, and I sometimes think about how intentionally devolving most online conversations into petty slapfights is one of the very effective astroturfing techniques. It's basically signal jamming anything substantive or cooperative because people get tired sifting through all the noise and get mad reading all the bad takes. Though I have no doubt that many of them are still 100% genuine foolish humans.
    • throwawayqqq11 11mo ago
      Image there is an objective truth to all debates and it shows, that one side is further away from it than the other. If that more wrong side was more capable of leaving its cult and admitting mistakes, the discourse would change its shape. The signal jamming as you called it, only works because signals get - wrongfully or not - reflected and amplified instead of absorbed.
• scriptbash 11mo ago
  > Note that nothing in the article is AI-specific This is such a tired counter argument against LLM safety concerns. You understand that persuasion and influence are behaviors on a spectrum. Meaning some people, or in this case products, are more or less or better or worse at persuading and influencing. In this case people are concerned with LLM's ability to influence more effectively than other modes that we have had in the past. For example, I have had many tech illiterate people tell me that they believe "AI" is 'intelligent' and 'knows everything' and trust its output without question. While at the same time I've yet to meet a single person who says the same thing about "targeted Facebook ads". So depressing watching all of you do free propo psy ops for these fascist corpos.
• bcrosby95 11mo ago
  Cost matters. Let's look at a piece of tech that literally changed humankind. The printing press. We could create copies of books before the printing press. All it did was reduce the cost.
  • AnimalMuppet 11mo ago
    That's an interesting example. We get a new technology, and cost goes down, and volume goes up, and it takes a couple generations for society to adjust. I think of it as the lower cost makes reaching people easier, which is like the gain going up. And in order for society to be able to function, people need to learn to turn their own, individual gain down - otherwise they get overwhelmed by the new volume of information, or by manipulation from those using the new medium.
    • pixl97 11mo ago
      Propaganda with books could start wars in decades. Propaganda with radio could start wars in years. Propaganda with TV could start wars in months. Propaganda with Internet/AI could start war in _____?
• _carbyau_ 11mo ago
  Come the next election, see how many people ask AI "who to vote for", and see whether each AI has a distinct suggestion...
• yehat 11mo ago
  Well well... recent "feature" of X revealing the actual "actors" location of operation shows how much "Russian troll armies" are there.. turns out there're rather overwhelming Indian and Bangladesh armies working hard for who? Common, say it! And despite of that, while cheap, not that cheaper compared to when the "agentic" approach enters the game.
  • notabee 11mo ago
    I really wish people would stop fixating on one nation-state or other entity when it comes to the astroturfing problem. It's something that's going to have all sorts of hands stirring the pot since it's basically just a very pernicious new form of marketing and propaganda. Any sizeable countries or corporations are going to be utilizing this new tool of manipulation, regardless of how scummy that may be.
• Nemo_bis 11mo ago
  The cheapest method by far is still TV networks. As a billionaire you can buy them without putting any of your own money, so it's effectively free. See Sinclair Broadcast Group and Paramount Skydance (Larry Ellison). As shown in "Network Propaganda", TV still influences all other media, including print media and social media, so you don't need to watch TV to be influenced.
• ajam1507 11mo ago
  "An atomic bomb is just a really big firecracker"