10 ms·

I've only read the abstract, but there is also plenty of evidence to suggest that people trust the output of LLMs more than other forms of media (or that they s

by SCdF 10mo ago

I've only read the abstract, but there is also plenty of evidence to suggest that people trust the output of LLMs more than other forms of media (or that they should). Partially because it feels like it comes from a place of authority, and partially because of how self confident AI always sounds. The LLM bot army stuff is concerning, sure. The real concern for me is incredibly rich people with no empathy for you or I, having interstitial control of that kind of messaging. See, all of the grok ai tweaks over the past however long.

prox 10mo ago
And just see all of history where totalitarians or despotic kings were in power.
andsoitis 10mo ago
Do you think these super wealthy people who control AI use the AI themselves? Do you think they are also “manipulated” by their own tool or do they, somehow, escape that capture?
- pjc50 10mo ago
  It's fairly clear from Twitter that it's possible to be a victim of your own system. But sycophancy has always been a problem for elites. It's very easy to surround yourselves with people who always say yes, and now you can have a machine do it too. This is how you get things like the colossal Facebook writeoff of "metaverse".
- wongarsu 10mo ago
  Isn't Grok just built as "the AI Elon Musk wants to use"? Starting from the goals of being "maximally truth seeking" and having no "woke" alignment and fewer safety rails, to the various "tweaks" to the Grok Twitter bot that happen to be related to Musk's world view Even Grok at one point looking up how Musk feels about a topic before answering fits that pattern. Not something that's healthy or that he would likely prefer when asked, but something that would produce answers that he personally likes when using it
  - andsoitis 10mo ago
    > Isn't Grok just built as "the AI Elon Musk wants to use"? No > Even Grok at one point looking up how Musk feels about a topic before answering fits that pattern. So it no longer does?
vintermann 10mo ago
People hate being manipulated. If you feel like you're being manipulated but you don't know by who or precisely what they want of you, then there's something of an instinct to get angry and lash out in unpredictable destructive ways. If nobody gets what they want, then at least the manipulators will regret messing with you. This is why social control won't work for long, no matter if AI supercharges it. We're already seeing the blowback from decades of advertising and public opinion shaping.
- pjc50 10mo ago
  People hate feeling manipulated, but they love propaganda that feeds their prejudices. People voluntarily turn on Fox News - even in public spaces - and get mad if you turn it off. Sufficiently effective propaganda produces its own cults. People want a sense of purpose and belonging. Sometimes even at the expense of their own lives, or (more easily) someone else's lives.
  - FridayoLeary 10mo ago
    [flagged]
    - ceejayoz 10mo ago
      > just smaller maybe This is like peak both-sidesism. You even openly describe the left’s equivalent of MAGA as “fringe”, FFS. One party’s former “fringe” is now in full control of it. And the country’s institutions.
      - FridayoLeary 10mo ago
        I was both siding in an effort to be as objective as possible. The truth is that i'm pretty dismayed at the current state of the Democrat party. Socialists like Mamdani and Sanders and the squad are way too powerful. People who are obsessed with tearing down cultural and social institutions and replacing them with performative identity politics and fabricated narratives are given platforms way bigger then they deserve. The worries of average Americans are dismissed. All those are issues that are tearing up the Democrat party from the inside. I can continue for hours but i don't want to start a flamewar of biblical proportions. So all i did was present the most balanced view i can muster and you still can't acknowledge that there might be truth in what i'm saying. The pendulum swings both ways. MSM has fallen victim to partisan politics. Something which Trump recognised and exploited back in 2015. Fox news is on the right, CNN, ABC et al is on the left.
eurleif 10mo ago
When I was visiting home last year, I noticed my mom would throw her dog's poop in random peoples' bushes after picking it up, instead of taking it with her in a bag. I told her she shouldn't do that, but she said she thought it was fine because people don't walk in bushes, and so they won't step in the poop. I did my best to explain to her that 1) kids play all kinds of places, including in bushes; 2) rain can spread it around into the rest of the person's yard; and 3) you need to respect other peoples' property even if you think it won't matter. She was unconvinced, but said she'd "think about my perspective" and "look it up" whether I was right. A few days later, she told me: "I asked AI and you were right about the dog poop". Really bizarre to me. I gave her the reasoning for why it's a bad thing to do, but she wouldn't accept it until she heard it from this "moral authority".
- auggierose 10mo ago
  Welcome to my world. People don't listen to reason or arguments, they only accept social proof / authority / money talks etc. And yes, AI is already an authority. Why do you think companies are spending so much money on it? For profit? No, for power, as then profit comes automatically.
- thymine_dimer 10mo ago
  Quite a tangent, but for the purpose of avoiding anaerobic decomposition (and byproducts, CH4, H2S etc) of the dog poo and associated compostable bag (if you’re in one of those neighbourhoods), I do the same as your mum. If possible, flick it off the path. Else use a bag. Nature is full of the faeces of plenty of other things which we don’t bother picking up.
  - rightbyte 10mo ago
    I hope you live in a sparsely populated area. If it wouldn't work if more people then you do it, it is not a good process.
    - thymine_dimer 10mo ago
      It’s a great process where I live. But you’re right. Doesn’t scale to populated areas. Wonder what the potential microbial turnover of lawn is? Multiply that by the average walk length and I bet that could handle one or two nuggets per day, even in a city. That’s a side hustle idea for any disengaged strava engineers. Leave me an acknowledgement on the ‘about’ page.
pjc50 10mo ago
> The real concern for me is incredibly rich people with no empathy for you or I, having interstitial control of that kind of messaging. See, all of the grok ai tweaks over the past however long. Indeed. It's always been clear to me that the "AI risk" people are looking in the wrong direction. All the AI risks are human risks, because we haven't solved "human alignment". An AI that's perfectly obedient to humans is still a huge risk when used as a force multiplier by a malevolent human. Any ""safeguards"" can easily be defeated with the Ender's Game approach.
- bananaflag 10mo ago
  I think people who care about superintelligent AI risk don't believe an AI that is subservient to humans is the solution to AI alignment, for exactly the same reasons as you. Stuff like Coherent Extrapolated Volition* (see the paper with this name) which focuses on what all mankind would want if they know more and they were smarter (or something like that) would be a way to go. *But Yudkowsky ditched CEV years ago, for reasons I don't understand (but I admit I haven't put in the effort to understand).
- ben_w 10mo ago
  More than one danger from any given tech can be true at the same time. Coal plants can produce local smog as well as global warming. There's certainly some AI risks that are the same as human risks, just as you say. But even though LLMs have very human failures (IMO because the models anthropomorphise themselves as part of their training, thus leading to the outward behaviours of our emotions and thus emit token sequences such as "I'm sorry" or "how embarrassing!" when they (probably) didn't actually create any internal structure that can have emotions like sorrow and embarrassment), that doesn't generalise to all AI. Any machine learning system that is given a poor quality fitness function to optimise, will optimise whatever that fitness function actually is, not what it was meant to be: "Literal minded genie" and "rules lawyering" may be well-worn tropes for good reason, likewise work-to-rule as a union tactic, but we've all seen how much more severe computers are at being literal-minded than humans.
- zahlman 10mo ago
  >An AI that's perfectly obedient to humans is still a huge risk when used as a force multiplier by a malevolent human. "Obedient" is anthropomorphizing too much (as there is no volition), but even then, it only matters according to how much agency the bot is extended. So there is also risk from neglectful humans who opt to present BS as fact due to an expectation of receiving fact and a failure to critique the BS.
rockskon 10mo ago
AI is wrong so often that anyone who routinely uses one will get burnt at some point. Users having unflinching trust in AI? I think not.
intended 10mo ago
>people trust the output of LLMs more than other Theres one paper I saw on this, which covered attitudes of teens. As I recall they were unaware of hallucinations. Do you have any other sources on hand?
sahilagarwal 10mo ago
I would go against the grain and say that LLMs take power away from incredibly rich people to shape mass preferences and give to the masses. Bot armies previously needed an army of humans to give responses on social media, which is incredibly tough to scale unless you have money and power. Now, that part is automated and scalable. So instead of only billionaires, someone with a 100K dollars could launch a small scale "campaign".
- WickyNilliams 10mo ago
  "someone with 100k dollars" is not exactly "the masses". It is a larger set, but it's just more rich/powerful people. Which I would not describe as the "masses". I know what you mean, but that descriptor seems off
  - sahilagarwal 10mo ago
    I kind of wonder how far would 200 dollars would go. Would it be enough to start a local campaign in a town of 100,000 people?
throwaway-0001 10mo ago
…Also partially because it’s better then most other sources
potato3732842 10mo ago
LLMs haven't been caught actively lying yet, which isn't something that can be said for anything else. Give it 5yr and their reputation will be in the toilet too.
- SCdF 10mo ago
  LLMs can't lie: they aren't alive. The text they produce contains lies, constantly, at almost every interaction.
  - potato3732842 10mo ago
    It's the technically true but incomplete or missing something things I'm worried about. Basically eventually it's gonna stop being "dumb wrong" and start being "evil person making a motivated argument in the comments" and "sleazy official press release politician speak" type wrong
    - hn_acc1 10mo ago
      Wasn't / isn't Grok already there? It already supported the "white genocide in SA" conspiracy theory at one point, AFAIK.
- ceejayoz 10mo ago
  > LLMs haven't been caught actively lying yet… Any time they say "I'm sorry" - which is very, very common - they're lying.
Noaidi 10mo ago
Exactly. On Facebook everyone is stupid. But this is AI, like in the movies! It is smarter than anyone! It is almost like AI in the movies was part of the plot to brainwash us into thinking LLM output is correct every time.
malshe 10mo ago
> Partially because it feels like it comes from a place of authority, and partially because of how self confident AI always sounds. To add to that, this research paper[1] argues that people with low AI literary are more receptive to AI messaging because they find it magical. The paper is now published but it's behind paywall so I shared the working paper link. [1] https://thearf-org-unified-admin.s3.amazonaws.com/MSI_Report_24-132.pdf https://thearf-org-unified-admin.s3.amazonaws.com/MSI_Report...
zahlman 10mo ago
When the LLMs output supposedly convincing BS that "people" (I assume you mean on average, not e.g. HN commentariat) trust, they aren't doing anything that's difficult for humans (assuming the humans already at least minimally understand the topic they're about to BS about). They're just doing it efficiently and shamelessly.
stocksinsmocks 10mo ago
The evening news was once a trusted source. Wikipedia had its run. Google too. Eventually, the weight of all the the thumbs on the scale will be felt and trust will be lost for good and then we will invent a new oracle.
theptip 10mo ago
I’ve seen this result. I wonder if it’s because LLMs are (grok notwithstanding) deliberately middle-of-the-road in their stances, and accurately and patiently report the facts? In which case a hypothetical liar LLM would not be as persuasive. Or is it because they are super-human already in some persuasion skills, and they can persuade people even of falsehoods?
inquirerGeneral 10mo ago
[dead]

by SCdF 10mo ago

prox 10mo ago

And just see all of history where totalitarians or despotic kings were in power.

andsoitis 10mo ago

Do you think these super wealthy people who control AI use the AI themselves? Do you think they are also “manipulated” by their own tool or do they, somehow, escape that capture?

pjc50 10mo ago
It's fairly clear from Twitter that it's possible to be a victim of your own system. But sycophancy has always been a problem for elites. It's very easy to surround yourselves with people who always say yes, and now you can have a machine do it too. This is how you get things like the colossal Facebook writeoff of "metaverse".
wongarsu 10mo ago
Isn't Grok just built as "the AI Elon Musk wants to use"? Starting from the goals of being "maximally truth seeking" and having no "woke" alignment and fewer safety rails, to the various "tweaks" to the Grok Twitter bot that happen to be related to Musk's world view Even Grok at one point looking up how Musk feels about a topic before answering fits that pattern. Not something that's healthy or that he would likely prefer when asked, but something that would produce answers that he personally likes when using it
- andsoitis 10mo ago
  > Isn't Grok just built as "the AI Elon Musk wants to use"? No > Even Grok at one point looking up how Musk feels about a topic before answering fits that pattern. So it no longer does?

vintermann 10mo ago

People hate being manipulated. If you feel like you're being manipulated but you don't know by who or precisely what they want of you, then there's something of an instinct to get angry and lash out in unpredictable destructive ways. If nobody gets what they want, then at least the manipulators will regret messing with you. This is why social control won't work for long, no matter if AI supercharges it. We're already seeing the blowback from decades of advertising and public opinion shaping.

pjc50 10mo ago
People hate feeling manipulated, but they love propaganda that feeds their prejudices. People voluntarily turn on Fox News - even in public spaces - and get mad if you turn it off. Sufficiently effective propaganda produces its own cults. People want a sense of purpose and belonging. Sometimes even at the expense of their own lives, or (more easily) someone else's lives.
- FridayoLeary 10mo ago
  [flagged]
  - ceejayoz 10mo ago
    > just smaller maybe This is like peak both-sidesism. You even openly describe the left’s equivalent of MAGA as “fringe”, FFS. One party’s former “fringe” is now in full control of it. And the country’s institutions.
    - FridayoLeary 10mo ago
      I was both siding in an effort to be as objective as possible. The truth is that i'm pretty dismayed at the current state of the Democrat party. Socialists like Mamdani and Sanders and the squad are way too powerful. People who are obsessed with tearing down cultural and social institutions and replacing them with performative identity politics and fabricated narratives are given platforms way bigger then they deserve. The worries of average Americans are dismissed. All those are issues that are tearing up the Democrat party from the inside. I can continue for hours but i don't want to start a flamewar of biblical proportions. So all i did was present the most balanced view i can muster and you still can't acknowledge that there might be truth in what i'm saying. The pendulum swings both ways. MSM has fallen victim to partisan politics. Something which Trump recognised and exploited back in 2015. Fox news is on the right, CNN, ABC et al is on the left.

eurleif 10mo ago

When I was visiting home last year, I noticed my mom would throw her dog's poop in random peoples' bushes after picking it up, instead of taking it with her in a bag. I told her she shouldn't do that, but she said she thought it was fine because people don't walk in bushes, and so they won't step in the poop. I did my best to explain to her that 1) kids play all kinds of places, including in bushes; 2) rain can spread it around into the rest of the person's yard; and 3) you need to respect other peoples' property even if you think it won't matter. She was unconvinced, but said she'd "think about my perspective" and "look it up" whether I was right. A few days later, she told me: "I asked AI and you were right about the dog poop". Really bizarre to me. I gave her the reasoning for why it's a bad thing to do, but she wouldn't accept it until she heard it from this "moral authority".

auggierose 10mo ago
Welcome to my world. People don't listen to reason or arguments, they only accept social proof / authority / money talks etc. And yes, AI is already an authority. Why do you think companies are spending so much money on it? For profit? No, for power, as then profit comes automatically.
thymine_dimer 10mo ago
Quite a tangent, but for the purpose of avoiding anaerobic decomposition (and byproducts, CH4, H2S etc) of the dog poo and associated compostable bag (if you’re in one of those neighbourhoods), I do the same as your mum. If possible, flick it off the path. Else use a bag. Nature is full of the faeces of plenty of other things which we don’t bother picking up.
- rightbyte 10mo ago
  I hope you live in a sparsely populated area. If it wouldn't work if more people then you do it, it is not a good process.
  - thymine_dimer 10mo ago
    It’s a great process where I live. But you’re right. Doesn’t scale to populated areas. Wonder what the potential microbial turnover of lawn is? Multiply that by the average walk length and I bet that could handle one or two nuggets per day, even in a city. That’s a side hustle idea for any disengaged strava engineers. Leave me an acknowledgement on the ‘about’ page.

pjc50 10mo ago

> The real concern for me is incredibly rich people with no empathy for you or I, having interstitial control of that kind of messaging. See, all of the grok ai tweaks over the past however long. Indeed. It's always been clear to me that the "AI risk" people are looking in the wrong direction. All the AI risks are human risks, because we haven't solved "human alignment". An AI that's perfectly obedient to humans is still a huge risk when used as a force multiplier by a malevolent human. Any ""safeguards"" can easily be defeated with the Ender's Game approach.

bananaflag 10mo ago
I think people who care about superintelligent AI risk don't believe an AI that is subservient to humans is the solution to AI alignment, for exactly the same reasons as you. Stuff like Coherent Extrapolated Volition* (see the paper with this name) which focuses on what all mankind would want if they know more and they were smarter (or something like that) would be a way to go. *But Yudkowsky ditched CEV years ago, for reasons I don't understand (but I admit I haven't put in the effort to understand).
ben_w 10mo ago
More than one danger from any given tech can be true at the same time. Coal plants can produce local smog as well as global warming. There's certainly some AI risks that are the same as human risks, just as you say. But even though LLMs have very human failures (IMO because the models anthropomorphise themselves as part of their training, thus leading to the outward behaviours of our emotions and thus emit token sequences such as "I'm sorry" or "how embarrassing!" when they (probably) didn't actually create any internal structure that can have emotions like sorrow and embarrassment), that doesn't generalise to all AI. Any machine learning system that is given a poor quality fitness function to optimise, will optimise whatever that fitness function actually is, not what it was meant to be: "Literal minded genie" and "rules lawyering" may be well-worn tropes for good reason, likewise work-to-rule as a union tactic, but we've all seen how much more severe computers are at being literal-minded than humans.
zahlman 10mo ago
>An AI that's perfectly obedient to humans is still a huge risk when used as a force multiplier by a malevolent human. "Obedient" is anthropomorphizing too much (as there is no volition), but even then, it only matters according to how much agency the bot is extended. So there is also risk from neglectful humans who opt to present BS as fact due to an expectation of receiving fact and a failure to critique the BS.

rockskon 10mo ago

AI is wrong so often that anyone who routinely uses one will get burnt at some point. Users having unflinching trust in AI? I think not.

intended 10mo ago

>people trust the output of LLMs more than other Theres one paper I saw on this, which covered attitudes of teens. As I recall they were unaware of hallucinations. Do you have any other sources on hand?

sahilagarwal 10mo ago

I would go against the grain and say that LLMs take power away from incredibly rich people to shape mass preferences and give to the masses. Bot armies previously needed an army of humans to give responses on social media, which is incredibly tough to scale unless you have money and power. Now, that part is automated and scalable. So instead of only billionaires, someone with a 100K dollars could launch a small scale "campaign".

WickyNilliams 10mo ago
"someone with 100k dollars" is not exactly "the masses". It is a larger set, but it's just more rich/powerful people. Which I would not describe as the "masses". I know what you mean, but that descriptor seems off
- sahilagarwal 10mo ago
  I kind of wonder how far would 200 dollars would go. Would it be enough to start a local campaign in a town of 100,000 people?

throwaway-0001 10mo ago

…Also partially because it’s better then most other sources

potato3732842 10mo ago

LLMs haven't been caught actively lying yet, which isn't something that can be said for anything else. Give it 5yr and their reputation will be in the toilet too.

SCdF 10mo ago
LLMs can't lie: they aren't alive. The text they produce contains lies, constantly, at almost every interaction.
- potato3732842 10mo ago
  It's the technically true but incomplete or missing something things I'm worried about. Basically eventually it's gonna stop being "dumb wrong" and start being "evil person making a motivated argument in the comments" and "sleazy official press release politician speak" type wrong
  - hn_acc1 10mo ago
    Wasn't / isn't Grok already there? It already supported the "white genocide in SA" conspiracy theory at one point, AFAIK.
ceejayoz 10mo ago
> LLMs haven't been caught actively lying yet… Any time they say "I'm sorry" - which is very, very common - they're lying.

Noaidi 10mo ago

Exactly. On Facebook everyone is stupid. But this is AI, like in the movies! It is smarter than anyone! It is almost like AI in the movies was part of the plot to brainwash us into thinking LLM output is correct every time.

malshe 10mo ago

> Partially because it feels like it comes from a place of authority, and partially because of how self confident AI always sounds. To add to that, this research paper[1] argues that people with low AI literary are more receptive to AI messaging because they find it magical. The paper is now published but it's behind paywall so I shared the working paper link. [1] https://thearf-org-unified-admin.s3.amazonaws.com/MSI_Report_24-132.pdf https://thearf-org-unified-admin.s3.amazonaws.com/MSI_Report...

zahlman 10mo ago

When the LLMs output supposedly convincing BS that "people" (I assume you mean on average, not e.g. HN commentariat) trust, they aren't doing anything that's difficult for humans (assuming the humans already at least minimally understand the topic they're about to BS about). They're just doing it efficiently and shamelessly.

stocksinsmocks 10mo ago

The evening news was once a trusted source. Wikipedia had its run. Google too. Eventually, the weight of all the the thumbs on the scale will be felt and trust will be lost for good and then we will invent a new oracle.

theptip 10mo ago

I’ve seen this result. I wonder if it’s because LLMs are (grok notwithstanding) deliberately middle-of-the-road in their stances, and accurately and patiently report the facts? In which case a hypothetical liar LLM would not be as persuasive. Or is it because they are super-human already in some persuasion skills, and they can persuade people even of falsehoods?

inquirerGeneral 10mo ago

[dead]