7 ms·

Hallucinations themselves are not even the greatest risk posed by LLMs. A much greater risk (in simple terms of probability times severity) I'd say is that chat

• tombert 2y ago
  I don't know if the model changed in the last six months, or maybe the wow factor has worn off a bit, but it also feels like ChatGPT has become a lot more "people-pleasy" than it was before. I'll ask it opinionated questions, and it will just do stuff to reaffirm what I said, even when I give contrary opinions in the same chat. I personally find it annoying (I don't really get along with human people pleasers either), but I could see someone using it as a tool to justify doing bad stuff, including self-harm; it doesn't really ever push back on what I say.
  • renewiltord 2y ago
    It's obvious, isn't it? The average Hacker News user, who has converged to the average Internet user, wants exactly that experience. LLMs are pretty good tools but perhaps they shouldn't be made available to others. People like me can use them but others seem to be killed when making contact. I think it's fine to restrict access to the elite. We don't let just anyone fly a fighter jet. Perhaps the average HN user should be protected from LLM interactions.
    • tombert 2y ago
      Is that really what you got from what I wrote? I wasn't suggesting that we restrict access to anyone, and I wasn't trying to imply that I'm somehow immune to the problems that were highlighted. I mentioned that I don't like people-pleasers and I find it a bit obnoxious when ChatGPT does it. I'm sure that there might be other bits of subtle encouragement it gives me that I don't notice, but I can't elaborate on those parts because, you know, I didn't notice them. I genuinely do not know how you got "we should restrict access" from my comment or the parent, you just extrapolated to make a pretty stupid joke.
      • renewiltord 2y ago
        Haha, I'm not claiming you're wanting that. I want that. So I'm saying it. What makes you think I was attempting to restate what you wrote?
        • tombert 2y ago
          It looked like you were being sarcastic, implying I was trying to suggest that I thought I was better than the average person in regards to handling AI. Particularly this line: > People like me can use them but others seem to be killed when making contact. If I misread that, fair enough.
          • renewiltord 2y ago
            Yeah, no, 100% sincere personal view. That guy who killed himself after using it is obviously not ready for this. Imagine killing yourself after typing in `print("Kill yourself")` at the Python REPL. The guy's an imbecile. We don't let just anyone drive a truck. I'm fine with nearly everyone being on the outside and unable to use these tools so long as I'm allowed to with as little trouble as possible. I recognize that the view that others should not be permitted things that I should be allowed to use is generally a sarcastically expressed view, but I genuinely think it has merit. Everyone who believes these things are dangerous and everyone to whom this is obviously dangerous, like the aforementioned mentally deficient individual, shouldn't be permitted use.
  • unclebucknasty 2y ago
    Yeah, I think it's coded to be super-conciliatory as some sort of apology for its hallucinations, but I find it annoying as well. Part of it is just like all automated prompts that try to be too human. When you know it's not human, it's almost patronizing and just annoying. But, it's actually worse, because it's generally apologizing for something completely wrong that it told you just moments before with extreme confidence.
  • taneq 2y ago
    I haven't played with it too much, and maybe it's changed recently or the paid version is different, but last week I found it irritatingly obtuse. > Me: Hi! Could you please help me find the problem with some code? > ChatGPT: Of course! Show me the code and I'll take a look! > Me: [bunch o' code] > ChatGPT: OK, it looks like you're trying to [do thing]. What did you want help with? > Me: I'm trying to find a problem with this code. > ChatGPT: Sure, just show me the code and I'll try to help! > Me: I just pasted it. > ChatGPT: I can't see it.
    • MrMcCall 2y ago
      Maybe they taught it that a) it doesn't work, and b) not to tell anyone. Lying will goad a person into trying again; the brutally honest truth will stop them like a brick wall.
• hexaga 2y ago
  More generally - AI that is good at convincing people is very powerful, and powerful things are dangerous. I'm increasingly coming around to the notion that AI tooling should have safety features concerned with not directly exposing humans to asymptotically increasing levels of 'convincingness' in generated output. Something like a weaker model used as a buffer. Projecting out to 5-10 years: what happens when LLMs are still producing hallucinatory semi-sense, but merely comprehending it makes the machine temporarily own you? A bit like getting hair caught in an angle grinder, that. Like most safety regulations, it'll take blood for the inking. Exposing mass numbers of people to these models strikes me as wildly negligent if we expect continued improvement along this axis.
  • kjs3 2y ago
    Yeah...this. I'm not so concerned that AI is going to put me out of a job or become Skynet. I'm concerned that people are offloading decision making and critical thinking to the AI, accepting it's response at face value and responding to concerns with "the AI said so...must be right". Companies are already maliciously exploiting this (e.g. the AI has denied your medical claim, and we can't tell you how it decided that because our models are trade secrets), but it will soon become de rigueur and people will think you're weird for questioning the wisdom of the AI.
    • Nevermark 2y ago
      The combination of blind faith in AI, and good faith highly informed understanding and agreement, achieved with help of AI, covers the full spectrum of the problem.
  • southernplaces7 2y ago
    >Projecting out to 5-10 years: what happens when LLMs are still producing hallucinatory semi-sense, but merely comprehending it makes the machine temporarily own you? A bit like getting hair caught in an angle grinder, that. Seriously? Do you suppose that it will pull this trick off through some sort of hypnotizing magic perhaps? I have a hard time imagining any sort of overly verbose, clause and condition-ridden chatbot convincing anyone of sound mind to seriously harm themselves or do some egregiously stupid/violent thing. The kinds of people who would be convinced by such "dangers" are likely to be mentally unstable or suggestible enough about it to in any case be convinced by any number of human beings anyhow. Aside from demonstrating the persistent AI woo that permeats many comments on this site, the logic above reminds me of the harping nonsense around the supposed dangers of video games or certain violent movies "making kinds do bad things", in years past. The prohibitionist nanny tendencies behind such fears are more dangerous than any silly chatbot AI..
• southernplaces7 2y ago
  In both of your linked examples, the people in question very likely had at least some sort of mental instability working in their minds. I have a hard time imagining any sort of overly verbose, clause and condition-ridden chatbot convincing anyone of sound mind to seriously harm themselves or do some egregiously stupid/violent thing. The kinds of people who would be convinced by such "harm dangers" are likely to be mentally unstable or suggestible enough about it to in any case be convinced by any number of human beings, or by books, or movies or any other sort of excuse for a mind that had problems well before seeing X or Y. By the logic of regulating AI for these supposed dangers, you could argue that literature, movie content, comic books, YouTube videos and that much loved boogeyman in previous years of violent video games should all be banned or regulated for the content they express. Such notions have a strongly nannyish, prohibitionist streak that's much more dangerous than some algorithm and the bullshit it spews to a few suggestible individuals. The media of course loves such narratives, because their breathless hysteria and contrived fear-mongering plays right into more eyeballs. Seeing people again take seriously such nonsense after idiocies like the media frenzy around video games in the early 2000s and prior to that, similar media fits about violent movies and even literature, is sort of sad. We don't need our tools for expression, and sources of information "regulated for harm" because a small minority of others can't get an easy grip on their psychological state.
  • skywhopper 2y ago
    Pretty much everyone has “some sort of mental instability working in their minds”.
    • southernplaces7 2y ago
      Don't be obtuse. There are degrees of mental instability and no, some random person having a touch of it in very specific ways isn't the same as someone deciding to try killing the Queen of England because a chatbot said so. Most people wouldn't be quite that deluded in that context. I'd love to see evidence of mental instability in "everyone" and its presence in many people is in any case no justification for what are in effect controls on freedom of speech and expression, just couched in a new boogeyman.
• zahlman 2y ago
  Is this somehow worse than humans talking each other into it?
  • skywhopper 2y ago
    Yes.
    • zahlman 2y ago
      How?
      • krupan 2y ago
        Does this really have to be spelled out?? Because a single human can only intimately converse with and convince a small number of people, while an LLM can do that with thousands (what is the upper limit even?) of people at a time. Also, because AI is being relentlessly marketed as being better than humans, thereby encouraging people to trust it even more than they might a fellow human.