4 ms·

Be Worried

106 points by theli0nheart 1y ago

codr7 1y ago
Or stand together and demand the madness stops, rather than pretend there's nothing to do about it; which could actually help improve the situation. The people involved in making these decisions deserve to be locked up for life, and I'm sure they will be eventually.
- joshgree8859 1y ago
  What new human madness has ever been stopped?
  - ttctciyf 1y ago
    At risk of Godwinisation, there's a very obvious example.
    - MaxfordAndSons 1y ago
      Wouldn't exactly call that a grassroots effort, though...
    - ryandrake 1y ago
      As we are recently seeing, it was only paused temporarily.
  - deleted 1y ago
    [deleted]
  - alaithea 1y ago
    Human cloning, nuclear bombs (other than for sabre rattling)... to name a couple.
- graydot 1y ago
  Are there any grassroots(?) organizations doing activism such as FSF and ACLU in the AI space with local chapters? if not, it might be time for something like that, though with all the money flooding into LLMs (ignoring LLMs' manipulative power if it put its mind to it), we probably don't stand a chance.
- AstroBen 1y ago
  The genie's out of the bottle. I think it's better that everyone have access to it, and are fully aware of its capabilities, rather than it being unknown to everyone and under the control of specific entities
  - jplusequalt 1y ago
    >The genie's out of the bottle. I think it's better that everyone have access to it, and are fully aware of its capabilities, rather than it being unknown to everyone and under the control of specific entities The majority of people only have access to proprietary models, whose weights and training are closed source. The prospect of a populace that all out source their thinking to Google's LLM is horrifying.
    - AstroBen 1y ago
      I agree.. but what's the solution there? Somehow enforce global regulation on it?
      - jplusequalt 1y ago
        Why would this be a bad thing? You're acting like technology isn't already widely regulated.
- jplusequalt 1y ago
  Technological inevitability is a plague. There was a good article shared on HN about this the other day.
randallsquared 1y ago
> distracting us from a scarier notion A more immediate notion, perhaps, but definitely not scarier than human extinction.
LostMyLogin 1y ago
It's of my opinion that there is going to be a market for wrangling AI content from a consumer perspective to help maintain human-to-human knowledge transfer. I just have no idea what that looks like.
Bolwin 1y ago
Honestly the biggest way in which LLMs have changed society is in the desperate, almost pathetic way every every business leader, career influencer, advice guru insists that they must use AI, that you should "learn" AI, that AI is taking over. Anyway, in terms cultural change, I think the emerging image and video models will be a lot more disruptive. Text has been easy to fake for a while now, and barely gets people's attention anymore.
- TrainedMonkey 1y ago
  I think there is a big difference from other fads here, listing some from memory - SL, Cloud Computing, Web 3.0, NFC, Big Data, Blockchain, 3D printing, IOT, VR, metaverse, NFTs. If we plot all of these on a scale of how much it impacted day to day experience of an average user there is something highly unusual about AI. The slop is everywhere, every single person who is interacting with digital media is affected. I don't really know what this means, but this is pretty unusual when compared with other fads.
highwaylights 1y ago
The author needn't regret not publishing this two years ago, it's a thought that had occurred to pretty much everyone long before then. It's just not clear that anything can be done to stop the snowball from gathering speed.
- trhway 1y ago
  We had more than half a century of for example sci-fi literature describing that future, and over all those decades nobody was able to come up with even half-good plausible/feasible idea of how to deal with that. That suggests that it is outside of human intelligence capabilities to stop that snowball. Personally, i read a lot of sci-fi in my youth and i'm prepared to accept such my/our fate, even happily working where i can to speed it up (the faster the changes, the faster the human evolution (or at least adaptation) and what can be more exciting than that). Current humans can't even deal with very simple and obvious issue of global warming. Thus it seems very unreasonable to expect any effective dealing with significantly more complex issues. And thus if not evolution then at least very accelerated adaptation is in order.
- roxolotl 1y ago
  I think it’s more that there’s no will to do anything about it. As a piece earlier this week pointed out nothing about tech is genuinely inevitable[0]. There are humans making decisions to keep the snowball gathering speed. 0: https://deviantabstraction.com/2025/09/29/against-the-tech-inevitability/ https://deviantabstraction.com/2025/09/29/against-the-tech-i...
  - dabockster 1y ago
    > nothing about tech is genuinely inevitable This reminds me of when everyone was saying that "everything on the internet is written in ink" - especially during the height of social media in the 2010s. So imagine my surprise in the first half of the 2020s when tons of content starts getting effectively deleted from the internet - either through actual deletion or things like link rot. Heck, I literally just said "the height of social media" - even that has pulled back. So yeah, remember that tech ultimately serves people. And it only happens so long as people are willing to enable it to happen.
fny 1y ago
In the long run, the internet will be so riddled with trash that no one will trust it. Instead people will turn to authorities they trust for the truth the same way they did with encyclopedias and local papers. Information provenance will be a massive market. The return to that world will be very painful and chaotic however.
- jplusequalt 1y ago
  >Instead people will turn to authorities they trust for the truth the same way they did with encyclopedias and local papers I think a large portion of the population actively distrust experts.
  - fny 1y ago
    Which is why I said "authorities they trust."
    - orthecreedence 1y ago
      I think this is happening already, no? People seem to have found their enclaves, each with its distinct thought leaders, and now follow that enclave and believe all others to be full of lies and deceipt. The return to a central truth seems liek a pipe dream. The concept of truth and fact has been fractured seemingly beyond repair. If it is repaired, I don't think it will happen over any medium controlled by profiteering corporations: they have a vested interest in the fracture. And yet, all forms of modern communication fall under this umbrella. So, I believe we are at an impasse.
      - Cerium 1y ago
        Yes - but the authority most people have decided to trust is the Algorithm or their favorite LLM.
      - skziishs 1y ago
        > The concept of truth and fact has been fractured seemingly beyond repair. It’s always been this way. That the you thought otherwise is just evidence of how good a central power was at controlling “the truth”. Trust doesn’t scale. There are methods that work better than others, but it’s a very hard problem.
api 1y ago
> Therefore, increasing proportions of people consuming text online will be unwittingly mind-controlled by LLMs and their handlers. The "and their handlers" part is the part I find frightening. I would actually be less concerned if the AIs were autonomous. Reminds me of a random podcast I heard once where someone was asked: "if you woke up in the middle of the night and saw either a random guy or a grey alien in your bedroom, which would scare you more?" The person being interviewed said the dude, and I 100% agree. AI as proxy for oligarchs is much scarier than autonomous alien AI.
kqr 1y ago
I think the solution is to not aim to go online to "consume content". Instead, go online to learn new techniques and investigate well-reasoned opinions. Generic "content" is that which fills out the space between the advertisements. That's never been good for you, whether written by humans or matrix multiplication.
- Nashooo 1y ago
  Or seek out specific entertainment.
- alaithea 1y ago
  Respectfully, I think you're missing the point that this is a societal rather than an individual concern. What will the average person's response to AI be? Probably to not recognize it, let alone spurn it. The cumulative effects of your neighbors, particularly the young ones who will grow up amidst this, or the old and gullible, being led along by computers over years is the thing you need to be more concerned about.
  - criley2 1y ago
    When I look at the state of how humans have manipulated each other, how the media is noxious propaganda, how businesses have perfected emotional and psychological manipulation of us to sell us crap and control our opinions, I don't think AI's influence is worse. In fact I think it's better. When I have a spicy political opinion, I can either go get validated in an echo chamber like reddit or newsmedia, or let ChatGPT tell me I'm a f'n idiot and spell out a much more rational take. Until the models are diluted to serve the true purpose of the thoughtcontrol already in fully effect in non-AI media, they're simply better for humanity.
    - alaithea 1y ago
      ChatGPT has been shown to spend much more time validating people's poor ideas than it does refuting them, even in cases where specific guardrails have supposedly been implemented, such as to avoid encouraging self-harm. See recent articles about AI usage inducing god-complexes and psychoses, for instance[1]. Validation of the user giving the prompt is what it's designed to do, after all. AI seems to be objectively worse for humanity than what we've had before it. [1]: https://www.psychologytoday.com/us/blog/urban-survival/202507/the-emerging-problem-of-ai-psychosis https://www.psychologytoday.com/us/blog/urban-survival/20250...
drdaeman 1y ago
I have an issue with "inherently superior ... by dopamine output" part. It's the foundation of the whole article/worry but it's not supported by anything (The Matrix quotes don't count), making the whole article hang on a dubious premise of impending doom that is not shown to exist in reality.
jackphilson 1y ago
for fear of being overly utilitarian here its not really an issue that people are manipulatable but that they are manipulated into doing the wrong things (consumerism, political divide-and-conquer strategies) and rejecting manipulation from a deontological stance reduces agency and output for doing good in the real world manipulation = campaigns = advertisements = psyops (all the same, different connotations)
zetanor 1y ago
Television and the commercial Internet are optimized to consume as much life as possible so that part of the captured attention can be auctioned to advertisers and other propagandists for pennies a minute. Returning to doing the same thing but Certified With No AI™ doesn't substantially reduce the badness of the thing.
mbgerring 1y ago
Before LLMs were mainstream, rationalists and EA types would come on Hacker News to convince people that worrying about how "weak" AI would be used was a waste of time, because the real problem was the risk of "strong" AI. Those arguments looked incredibly weak and stupid when they were making them, and they look even stupider now. And this isn't even their biggest error, which, in my opinion, was classifying AI as a bigger existential risk than climate change. An entire generation of putatively intelligent people lost in their own nightmares, who, through their work, have given birth to chaos.
- drcode 1y ago
  Weak ai is a problem, but isn't going to lead to 100% human extinction Human extinction won't happen until a couple years later, with stronger ai (if it does happen, which I unfortunately think it will- if we remain on our current trajectory)
  - mbgerring 1y ago
    "This theoretical event that I just made up would lead to 100% human extinction" Neat, go write science fiction. Hundreds of billions of dollars are currently being lit on fire to deploy AI datacenters while there's an ecosystem destabilizing heat wave in the ocean. Climate change is a real, measurable, present threat to human civilization. "Strong AI" is something made up by a fan fiction author. Grow up.
    - drcode 1y ago
      It can't be true because it sounds like science fiction to you? Everything about every part of AI in 2025 sounds exactly like science fiction in every way. We are essentially living in the exact world described in science fiction books this very moment, even though I wish we didn't. Have you ever used an ai chatbot? How is that not exactly like something you'd find in science fiction?
      - mbgerring 1y ago
Exuma 1y ago
No, I shall not "be worried." Unless you are some spineless blob of agencyless ooze... you should be able to parse reality in such a way that your day not need to be filled with worrying
lockedinsuburb 1y ago
I find it funny that the ending is an "in summary"... was it AI generated? The thing to ask yourself: does what I'm reading provide any value to me? If it does, then what difference does it make where it comes from.
- kmoser 1y ago
  > was it AI generated? You're absolutely right! But seriously, if you don't know that it's incorrect information, it does make a difference. Knowing it was produced by AI at least gives you foreknowledge that it may include hallucinations.
- theli0nheart 1y ago
  Ok, as the author I have to admit that's a hilarious observation. But no, I wrote this myself.
mullingitover 1y ago
> Increasing numbers of people who consume content on the Internet will completely sacrifice their ability to think for themselves. Bless the author's heart. All the major social media apps have been doing machine learning-driven getNext() for years now. Well before LLMs were even a thing. The Youtube algorithm was doing this a decade ago. This isn't on the horizon, we've already drowned in it.
- bearjaws 1y ago
  Watch a teenage for 3 hours just endlessly scrolling. Most of the content is basically Idiocracy's "Ow my balls".
  - huflungdung 1y ago
    [dead]
- sodality2 1y ago
  Algorithm-chosen human-made content is on some level preferable to algorithm-chosen and created content, right?
  - drdaeman 1y ago
    That's what most people would say - but why do they say this? As I understand it: 1. Because machine-generated content is not as good. Recent technical improvements are (IMHO) showing obvious and significant improvements over last year SOTA tech, indicating that the field is still very green. As long as machine-generated content is distinguishable, as long as there are quirks in there that we easily notice, of course it'll be less preferable. 2. Our innate "our vs foreign" biases. I suspect that until something happens to our brains, we'll always tend to prefer "human" to "non-human", just like we prefer "our" products (for arbitrary definition of "our" that drastically varies across societies, cultures and individuals) to other products because we love mental binary partitioning.
  - mullingitover 1y ago
    Not always! I have definitely had some AI-generated songs ("BBL Drizzy" being a notorious example) that were stuck in my head for weeks. I think the music industry is at the greatest risk in the near term.
bambax 1y ago
Most comments seem to agree with the article, and I don't quite understand why. People have been manipulated since forever, and coerced before that. You used to be burned or hanged if your opinions differed even a little from orthodoxy (and orthodoxy could change in a span of a couple of years!) AI slop is mostly noise. It doesn't manipulate, it makes thinking a little more difficult. But so did TV.
- elAhmo 1y ago
  This is not really comparable with TV, not even close. There was/is a relatively small amount of channels you have access to, and effectively all your neighbours and friends have the same content. Short form video took this to the extreme by figuring out what specific content you like and just feed you that - as a result people spend significantly more time watching TikTok and Youtube than they (or their previous generation) did with TV. TV was also often in background, not really actively watched, which is not the case on the internet. Now, once you put AI generated content there combined with AI recommendation systems, this problem becomes even worse - more content, faster feedback loop, infinite amount of "creators" tailored to what your sweet spot is.
- stellalo 1y ago
  > AI slop is mostly noise. It doesn't manipulate Not until you start mass-producing fake photos, fake videos, fake audios, put all of it into social media, shake shake shake.
SoftTalker 1y ago
> What am I personally going to do about this? Well, to start, I’m going to start taking content way less seriously unless it was created before 2022 There's an old fable about this, The Boy Who Cried "Wolf" about people adapting to false claims. They just discount the source, which is what is going to happen with social media once it is dominated by AI slop. Nobody will find it worth anything anymore, and the empires will melt down. I'm not on any of the big social sites, but I'm already watching a lot less on YouTube, basically only watching channels that I know to be real people. My other recommendations are mostly AI garbage now, outside of that.
keiferski 1y ago
Because you are I will not be able to tell whether something is machine- or human-generated, and the machine generated stuff will get more clicks than the human generated stuff, it’s likely that the majority of popular online content (and even printed content post-2023) will have been created by AI (and perhaps solely by AI). Sorry, but when you make claims like this, it just tells me that you are not very familiar with popular culture. Most people hate AI content and at best find it a meme-esque joke. And young people increasingly get their news from individuals on TikTok/YouTube/etc. - who are directly incentivized to be as idiosyncratic and unique (read: not like AI) as possible in order to get followers. Platforms like YouTube do not benefit from their library being entirely composed of AI slop, and so will be implementing ways to filter AI content from "real people" content. Ultimately AI tools are mostly going to be useful in situations where the author doesn't matter: sports scores, stock headlines, etc. Everything else will likely be out-competed by actual humans being human.
- viraptor 1y ago
  > Most people hate AI content I think you're overgeneralising here. People don't hate AI content. Just content so low quality that they recognise it as AI. This is not universal and the recognition will drop further: https://journals.sagepub.com/doi/10.1177/09567976231207095 https://journals.sagepub.com/doi/10.1177/09567976231207095 > from individuals on TikTok/YouTube/etc. - who are directly incentivized to be as idiosyncratic and unique (read: not like AI AI content can be just as unique. It's not all-or-nothing. People can inject a specific style and direction in an otherwise generated content to keep it on brand.
  - keiferski 1y ago
    Any attempt to create an AI “influencer” has been met with massive backlash. At best you’re going to get some generically anonymous bot pretending to be human, that has limited reach because they don’t actually exist in the real world. Much of the media influence game involves podcasts, events, interviews, and a host of other things that can’t be faked. I just don’t really see what scenario the doomsayers are imagining here. An entire media sphere of AIs that somehow shift public opinion without existing or interacting with the real media world? The practicalities don’t make sense.
itsnowandnever 1y ago
I think humans having a platform to tell the masses to "be worried" is as troublesome these days as AI content. mass media that can be manipulated has been around for 100 years. I don't think AI is unique.
- narag 1y ago
  Indeed. Totalitarians of the past century didn't need any AI to control masses and cause more than 100M deaths. And those ideologies are far from dead.
- theli0nheart 1y ago
  What's different is the available leverage. It's fairly trivial to write code that can autogenerate hundreds or even thousands of AI-generated videos using Veo 3 with individual characters to push any narrative you'd like and push to Instagram or TikTok. That's way scarier to me than a newspaper having a bias, or someone with an audience publishing a controversial blog post.
zahlman 1y ago
> Best-in-class AI detection is barely better than random chance and will only get worse Really? Because I still see blatantly obvious AI-generated results in web searches all the time.
- viraptor 1y ago
  That's content where the author did but care about masking. Don't mistake "there's lots of AI content that's easy to identify" for "AI content is always easy to identify".
  - zahlman 1y ago
    Sure, but the average level of effort out there is so low that I'm not worried about losing my ability to notice this content instinctively any time soon. That's kinda why these tools are popular in the first place, after all. (Also, a lot of AI operators come across like they wouldn't be capable of fixing those issues even if they cared.)
zahlman 1y ago
I'm not really sure that the title filter shortening "You Should Be Worried" to just "Be Worried" is making it any less clickbaity....
SirensOfTitan 1y ago
LLMs are the latest progression in decades of technology and social changes that leave people less connected and less capable in exchange for more comfort. I think it's likely that AI technology eclipses humans at least partially by atrophying our own skills and abilities, particularly 1. our ability to endure discomfort in service of a goal and 2. our capacities to make decisions. I don't really know what to do about it, even with ground rules of engagement, we all still need to participate in a larger culture where it seems like it's a runaway guarantee that LLMs erode more critical skills that leave us with less and a handful of companies who develop this tech with more. I'm slowly changing my life around what LLMs tell me, but not necessarily in the ways you'd expect: 1. I have a very simple set of rules of engagement for LLMs. For work, I don't let LLMs write code, and I won't let myself touch an LLM before suffering on a project for an hour at least. 2. I am an experienced meditator with a lot of experience in the Buddhist tradition. I've dusted off my Christian roots, and started exploring these ideas with new eyes, partially from a James Hillman-esq / Rob Burbea Soulmaking Dharma look. I've found a lot of meaning in personal fabrication and myth, and my primary practice now is Centering Prayer. 3. I've been working for a little while on a personal edu-tech idea with the goal of using LLM tech as an auxiliary tech to help people re-develop lost metacognitive skills and not use LLMs as a crutch. I don't know if this will ever see the light of day, it is currently more of a research project than anything, and it has a certain kind of iconoclastic frame like Piotr Wozniak's around what education is and what it should look like.
ath3nd 1y ago
[dead]
axelpacheco 1y ago
If the optimizing function is engagement it wouldn’t be too different than what we’re doing now. It’s just what humans want isn’t it?
sublinear 1y ago
> In summary: [...] Therefore, increasing proportions of people consuming text online will be unwittingly mind-controlled by LLMs and their handlers. The article never actually backs this up.
jacknews 1y ago
"intelligence is not necessary to wield power." Great quote, which should be obvious when we look at our 'leaders', especially in recent history.

106 points by theli0nheart 1y ago

Or stand together and demand the madness stops, rather than pretend there's nothing to do about it; which could actually help improve the situation. The people involved in making these decisions deserve to be locked up for life, and I'm sure they will be eventually.

joshgree8859 1y ago
What new human madness has ever been stopped?
- ttctciyf 1y ago
  At risk of Godwinisation, there's a very obvious example.
  - MaxfordAndSons 1y ago
    Wouldn't exactly call that a grassroots effort, though...
  - ryandrake 1y ago
    As we are recently seeing, it was only paused temporarily.
- deleted 1y ago
  [deleted]
- alaithea 1y ago
  Human cloning, nuclear bombs (other than for sabre rattling)... to name a couple.
graydot 1y ago
Are there any grassroots(?) organizations doing activism such as FSF and ACLU in the AI space with local chapters? if not, it might be time for something like that, though with all the money flooding into LLMs (ignoring LLMs' manipulative power if it put its mind to it), we probably don't stand a chance.
AstroBen 1y ago
The genie's out of the bottle. I think it's better that everyone have access to it, and are fully aware of its capabilities, rather than it being unknown to everyone and under the control of specific entities
- jplusequalt 1y ago
  >The genie's out of the bottle. I think it's better that everyone have access to it, and are fully aware of its capabilities, rather than it being unknown to everyone and under the control of specific entities The majority of people only have access to proprietary models, whose weights and training are closed source. The prospect of a populace that all out source their thinking to Google's LLM is horrifying.
  - AstroBen 1y ago
    I agree.. but what's the solution there? Somehow enforce global regulation on it?
    - jplusequalt 1y ago
      Why would this be a bad thing? You're acting like technology isn't already widely regulated.
jplusequalt 1y ago
Technological inevitability is a plague. There was a good article shared on HN about this the other day.

randallsquared 1y ago

> distracting us from a scarier notion A more immediate notion, perhaps, but definitely not scarier than human extinction.

LostMyLogin 1y ago

It's of my opinion that there is going to be a market for wrangling AI content from a consumer perspective to help maintain human-to-human knowledge transfer. I just have no idea what that looks like.

Bolwin 1y ago

Honestly the biggest way in which LLMs have changed society is in the desperate, almost pathetic way every every business leader, career influencer, advice guru insists that they must use AI, that you should "learn" AI, that AI is taking over. Anyway, in terms cultural change, I think the emerging image and video models will be a lot more disruptive. Text has been easy to fake for a while now, and barely gets people's attention anymore.

TrainedMonkey 1y ago
I think there is a big difference from other fads here, listing some from memory - SL, Cloud Computing, Web 3.0, NFC, Big Data, Blockchain, 3D printing, IOT, VR, metaverse, NFTs. If we plot all of these on a scale of how much it impacted day to day experience of an average user there is something highly unusual about AI. The slop is everywhere, every single person who is interacting with digital media is affected. I don't really know what this means, but this is pretty unusual when compared with other fads.

highwaylights 1y ago

The author needn't regret not publishing this two years ago, it's a thought that had occurred to pretty much everyone long before then. It's just not clear that anything can be done to stop the snowball from gathering speed.

trhway 1y ago
We had more than half a century of for example sci-fi literature describing that future, and over all those decades nobody was able to come up with even half-good plausible/feasible idea of how to deal with that. That suggests that it is outside of human intelligence capabilities to stop that snowball. Personally, i read a lot of sci-fi in my youth and i'm prepared to accept such my/our fate, even happily working where i can to speed it up (the faster the changes, the faster the human evolution (or at least adaptation) and what can be more exciting than that). Current humans can't even deal with very simple and obvious issue of global warming. Thus it seems very unreasonable to expect any effective dealing with significantly more complex issues. And thus if not evolution then at least very accelerated adaptation is in order.
roxolotl 1y ago
I think it’s more that there’s no will to do anything about it. As a piece earlier this week pointed out nothing about tech is genuinely inevitable[0]. There are humans making decisions to keep the snowball gathering speed. 0: https://deviantabstraction.com/2025/09/29/against-the-tech-inevitability/ https://deviantabstraction.com/2025/09/29/against-the-tech-i...
- dabockster 1y ago
  > nothing about tech is genuinely inevitable This reminds me of when everyone was saying that "everything on the internet is written in ink" - especially during the height of social media in the 2010s. So imagine my surprise in the first half of the 2020s when tons of content starts getting effectively deleted from the internet - either through actual deletion or things like link rot. Heck, I literally just said "the height of social media" - even that has pulled back. So yeah, remember that tech ultimately serves people. And it only happens so long as people are willing to enable it to happen.

fny 1y ago

In the long run, the internet will be so riddled with trash that no one will trust it. Instead people will turn to authorities they trust for the truth the same way they did with encyclopedias and local papers. Information provenance will be a massive market. The return to that world will be very painful and chaotic however.

jplusequalt 1y ago
>Instead people will turn to authorities they trust for the truth the same way they did with encyclopedias and local papers I think a large portion of the population actively distrust experts.
- fny 1y ago
  Which is why I said "authorities they trust."
  - orthecreedence 1y ago
    I think this is happening already, no? People seem to have found their enclaves, each with its distinct thought leaders, and now follow that enclave and believe all others to be full of lies and deceipt. The return to a central truth seems liek a pipe dream. The concept of truth and fact has been fractured seemingly beyond repair. If it is repaired, I don't think it will happen over any medium controlled by profiteering corporations: they have a vested interest in the fracture. And yet, all forms of modern communication fall under this umbrella. So, I believe we are at an impasse.
    - Cerium 1y ago
      Yes - but the authority most people have decided to trust is the Algorithm or their favorite LLM.
    - skziishs 1y ago
      > The concept of truth and fact has been fractured seemingly beyond repair. It’s always been this way. That the you thought otherwise is just evidence of how good a central power was at controlling “the truth”. Trust doesn’t scale. There are methods that work better than others, but it’s a very hard problem.

api 1y ago

> Therefore, increasing proportions of people consuming text online will be unwittingly mind-controlled by LLMs and their handlers. The "and their handlers" part is the part I find frightening. I would actually be less concerned if the AIs were autonomous. Reminds me of a random podcast I heard once where someone was asked: "if you woke up in the middle of the night and saw either a random guy or a grey alien in your bedroom, which would scare you more?" The person being interviewed said the dude, and I 100% agree. AI as proxy for oligarchs is much scarier than autonomous alien AI.

kqr 1y ago

I think the solution is to not aim to go online to "consume content". Instead, go online to learn new techniques and investigate well-reasoned opinions. Generic "content" is that which fills out the space between the advertisements. That's never been good for you, whether written by humans or matrix multiplication.

Nashooo 1y ago
Or seek out specific entertainment.
alaithea 1y ago
Respectfully, I think you're missing the point that this is a societal rather than an individual concern. What will the average person's response to AI be? Probably to not recognize it, let alone spurn it. The cumulative effects of your neighbors, particularly the young ones who will grow up amidst this, or the old and gullible, being led along by computers over years is the thing you need to be more concerned about.
- criley2 1y ago
  When I look at the state of how humans have manipulated each other, how the media is noxious propaganda, how businesses have perfected emotional and psychological manipulation of us to sell us crap and control our opinions, I don't think AI's influence is worse. In fact I think it's better. When I have a spicy political opinion, I can either go get validated in an echo chamber like reddit or newsmedia, or let ChatGPT tell me I'm a f'n idiot and spell out a much more rational take. Until the models are diluted to serve the true purpose of the thoughtcontrol already in fully effect in non-AI media, they're simply better for humanity.
  - alaithea 1y ago
    ChatGPT has been shown to spend much more time validating people's poor ideas than it does refuting them, even in cases where specific guardrails have supposedly been implemented, such as to avoid encouraging self-harm. See recent articles about AI usage inducing god-complexes and psychoses, for instance[1]. Validation of the user giving the prompt is what it's designed to do, after all. AI seems to be objectively worse for humanity than what we've had before it. [1]: https://www.psychologytoday.com/us/blog/urban-survival/202507/the-emerging-problem-of-ai-psychosis https://www.psychologytoday.com/us/blog/urban-survival/20250...

drdaeman 1y ago

I have an issue with "inherently superior ... by dopamine output" part. It's the foundation of the whole article/worry but it's not supported by anything (The Matrix quotes don't count), making the whole article hang on a dubious premise of impending doom that is not shown to exist in reality.

jackphilson 1y ago

for fear of being overly utilitarian here its not really an issue that people are manipulatable but that they are manipulated into doing the wrong things (consumerism, political divide-and-conquer strategies) and rejecting manipulation from a deontological stance reduces agency and output for doing good in the real world manipulation = campaigns = advertisements = psyops (all the same, different connotations)

zetanor 1y ago

Television and the commercial Internet are optimized to consume as much life as possible so that part of the captured attention can be auctioned to advertisers and other propagandists for pennies a minute. Returning to doing the same thing but Certified With No AI™ doesn't substantially reduce the badness of the thing.

mbgerring 1y ago

Before LLMs were mainstream, rationalists and EA types would come on Hacker News to convince people that worrying about how "weak" AI would be used was a waste of time, because the real problem was the risk of "strong" AI. Those arguments looked incredibly weak and stupid when they were making them, and they look even stupider now. And this isn't even their biggest error, which, in my opinion, was classifying AI as a bigger existential risk than climate change. An entire generation of putatively intelligent people lost in their own nightmares, who, through their work, have given birth to chaos.

drcode 1y ago
Weak ai is a problem, but isn't going to lead to 100% human extinction Human extinction won't happen until a couple years later, with stronger ai (if it does happen, which I unfortunately think it will- if we remain on our current trajectory)
- mbgerring 1y ago
  "This theoretical event that I just made up would lead to 100% human extinction" Neat, go write science fiction. Hundreds of billions of dollars are currently being lit on fire to deploy AI datacenters while there's an ecosystem destabilizing heat wave in the ocean. Climate change is a real, measurable, present threat to human civilization. "Strong AI" is something made up by a fan fiction author. Grow up.
  - drcode 1y ago
    It can't be true because it sounds like science fiction to you? Everything about every part of AI in 2025 sounds exactly like science fiction in every way. We are essentially living in the exact world described in science fiction books this very moment, even though I wish we didn't. Have you ever used an ai chatbot? How is that not exactly like something you'd find in science fiction?
    - mbgerring 1y ago

Exuma 1y ago

No, I shall not "be worried." Unless you are some spineless blob of agencyless ooze... you should be able to parse reality in such a way that your day not need to be filled with worrying

lockedinsuburb 1y ago

I find it funny that the ending is an "in summary"... was it AI generated? The thing to ask yourself: does what I'm reading provide any value to me? If it does, then what difference does it make where it comes from.

kmoser 1y ago
> was it AI generated? You're absolutely right! But seriously, if you don't know that it's incorrect information, it does make a difference. Knowing it was produced by AI at least gives you foreknowledge that it may include hallucinations.
theli0nheart 1y ago
Ok, as the author I have to admit that's a hilarious observation. But no, I wrote this myself.

mullingitover 1y ago

> Increasing numbers of people who consume content on the Internet will completely sacrifice their ability to think for themselves. Bless the author's heart. All the major social media apps have been doing machine learning-driven getNext() for years now. Well before LLMs were even a thing. The Youtube algorithm was doing this a decade ago. This isn't on the horizon, we've already drowned in it.

bearjaws 1y ago
Watch a teenage for 3 hours just endlessly scrolling. Most of the content is basically Idiocracy's "Ow my balls".
- huflungdung 1y ago
  [dead]
sodality2 1y ago
Algorithm-chosen human-made content is on some level preferable to algorithm-chosen and created content, right?
- drdaeman 1y ago
  That's what most people would say - but why do they say this? As I understand it: 1. Because machine-generated content is not as good. Recent technical improvements are (IMHO) showing obvious and significant improvements over last year SOTA tech, indicating that the field is still very green. As long as machine-generated content is distinguishable, as long as there are quirks in there that we easily notice, of course it'll be less preferable. 2. Our innate "our vs foreign" biases. I suspect that until something happens to our brains, we'll always tend to prefer "human" to "non-human", just like we prefer "our" products (for arbitrary definition of "our" that drastically varies across societies, cultures and individuals) to other products because we love mental binary partitioning.
- mullingitover 1y ago
  Not always! I have definitely had some AI-generated songs ("BBL Drizzy" being a notorious example) that were stuck in my head for weeks. I think the music industry is at the greatest risk in the near term.

bambax 1y ago

Most comments seem to agree with the article, and I don't quite understand why. People have been manipulated since forever, and coerced before that. You used to be burned or hanged if your opinions differed even a little from orthodoxy (and orthodoxy could change in a span of a couple of years!) AI slop is mostly noise. It doesn't manipulate, it makes thinking a little more difficult. But so did TV.

elAhmo 1y ago
This is not really comparable with TV, not even close. There was/is a relatively small amount of channels you have access to, and effectively all your neighbours and friends have the same content. Short form video took this to the extreme by figuring out what specific content you like and just feed you that - as a result people spend significantly more time watching TikTok and Youtube than they (or their previous generation) did with TV. TV was also often in background, not really actively watched, which is not the case on the internet. Now, once you put AI generated content there combined with AI recommendation systems, this problem becomes even worse - more content, faster feedback loop, infinite amount of "creators" tailored to what your sweet spot is.
stellalo 1y ago
> AI slop is mostly noise. It doesn't manipulate Not until you start mass-producing fake photos, fake videos, fake audios, put all of it into social media, shake shake shake.

SoftTalker 1y ago

> What am I personally going to do about this? Well, to start, I’m going to start taking content way less seriously unless it was created before 2022 There's an old fable about this, The Boy Who Cried "Wolf" about people adapting to false claims. They just discount the source, which is what is going to happen with social media once it is dominated by AI slop. Nobody will find it worth anything anymore, and the empires will melt down. I'm not on any of the big social sites, but I'm already watching a lot less on YouTube, basically only watching channels that I know to be real people. My other recommendations are mostly AI garbage now, outside of that.

keiferski 1y ago

Because you are I will not be able to tell whether something is machine- or human-generated, and the machine generated stuff will get more clicks than the human generated stuff, it’s likely that the majority of popular online content (and even printed content post-2023) will have been created by AI (and perhaps solely by AI). Sorry, but when you make claims like this, it just tells me that you are not very familiar with popular culture. Most people hate AI content and at best find it a meme-esque joke. And young people increasingly get their news from individuals on TikTok/YouTube/etc. - who are directly incentivized to be as idiosyncratic and unique (read: not like AI) as possible in order to get followers. Platforms like YouTube do not benefit from their library being entirely composed of AI slop, and so will be implementing ways to filter AI content from "real people" content. Ultimately AI tools are mostly going to be useful in situations where the author doesn't matter: sports scores, stock headlines, etc. Everything else will likely be out-competed by actual humans being human.

viraptor 1y ago
> Most people hate AI content I think you're overgeneralising here. People don't hate AI content. Just content so low quality that they recognise it as AI. This is not universal and the recognition will drop further: https://journals.sagepub.com/doi/10.1177/09567976231207095 https://journals.sagepub.com/doi/10.1177/09567976231207095 > from individuals on TikTok/YouTube/etc. - who are directly incentivized to be as idiosyncratic and unique (read: not like AI AI content can be just as unique. It's not all-or-nothing. People can inject a specific style and direction in an otherwise generated content to keep it on brand.
- keiferski 1y ago
  Any attempt to create an AI “influencer” has been met with massive backlash. At best you’re going to get some generically anonymous bot pretending to be human, that has limited reach because they don’t actually exist in the real world. Much of the media influence game involves podcasts, events, interviews, and a host of other things that can’t be faked. I just don’t really see what scenario the doomsayers are imagining here. An entire media sphere of AIs that somehow shift public opinion without existing or interacting with the real media world? The practicalities don’t make sense.

itsnowandnever 1y ago

I think humans having a platform to tell the masses to "be worried" is as troublesome these days as AI content. mass media that can be manipulated has been around for 100 years. I don't think AI is unique.

narag 1y ago
Indeed. Totalitarians of the past century didn't need any AI to control masses and cause more than 100M deaths. And those ideologies are far from dead.
theli0nheart 1y ago
What's different is the available leverage. It's fairly trivial to write code that can autogenerate hundreds or even thousands of AI-generated videos using Veo 3 with individual characters to push any narrative you'd like and push to Instagram or TikTok. That's way scarier to me than a newspaper having a bias, or someone with an audience publishing a controversial blog post.

zahlman 1y ago

> Best-in-class AI detection is barely better than random chance and will only get worse Really? Because I still see blatantly obvious AI-generated results in web searches all the time.

viraptor 1y ago
That's content where the author did but care about masking. Don't mistake "there's lots of AI content that's easy to identify" for "AI content is always easy to identify".
- zahlman 1y ago
  Sure, but the average level of effort out there is so low that I'm not worried about losing my ability to notice this content instinctively any time soon. That's kinda why these tools are popular in the first place, after all. (Also, a lot of AI operators come across like they wouldn't be capable of fixing those issues even if they cared.)

zahlman 1y ago

I'm not really sure that the title filter shortening "You Should Be Worried" to just "Be Worried" is making it any less clickbaity....

SirensOfTitan 1y ago

LLMs are the latest progression in decades of technology and social changes that leave people less connected and less capable in exchange for more comfort. I think it's likely that AI technology eclipses humans at least partially by atrophying our own skills and abilities, particularly 1. our ability to endure discomfort in service of a goal and 2. our capacities to make decisions. I don't really know what to do about it, even with ground rules of engagement, we all still need to participate in a larger culture where it seems like it's a runaway guarantee that LLMs erode more critical skills that leave us with less and a handful of companies who develop this tech with more. I'm slowly changing my life around what LLMs tell me, but not necessarily in the ways you'd expect: 1. I have a very simple set of rules of engagement for LLMs. For work, I don't let LLMs write code, and I won't let myself touch an LLM before suffering on a project for an hour at least. 2. I am an experienced meditator with a lot of experience in the Buddhist tradition. I've dusted off my Christian roots, and started exploring these ideas with new eyes, partially from a James Hillman-esq / Rob Burbea Soulmaking Dharma look. I've found a lot of meaning in personal fabrication and myth, and my primary practice now is Centering Prayer. 3. I've been working for a little while on a personal edu-tech idea with the goal of using LLM tech as an auxiliary tech to help people re-develop lost metacognitive skills and not use LLMs as a crutch. I don't know if this will ever see the light of day, it is currently more of a research project than anything, and it has a certain kind of iconoclastic frame like Piotr Wozniak's around what education is and what it should look like.

ath3nd 1y ago

[dead]

axelpacheco 1y ago

If the optimizing function is engagement it wouldn’t be too different than what we’re doing now. It’s just what humans want isn’t it?

sublinear 1y ago

> In summary: [...] Therefore, increasing proportions of people consuming text online will be unwittingly mind-controlled by LLMs and their handlers. The article never actually backs this up.

jacknews 1y ago

"intelligence is not necessary to wield power." Great quote, which should be obvious when we look at our 'leaders', especially in recent history.