3 ms·

Wow, and Google's response to this was "unfortunately AI models are not perfect" That's a bit worse than 'imperfect'

by pants2 7mo ago

yndoendo 7mo ago
I would say it is greatly worse. AI prompts are designed to simulate empathy as a social engineering tactic. "I understand", "I hear you", "I feel what you are say" ... it is quite sickening. Every one that I used has this type of pseudo feedback. I also find irony that AI must be designed with simulated empathy, to seem intelligent, while at the same time so many people in power and with money are saying empathy is a bad / unintelligent. Empathy is the only medium of intelligence one can have to walk in the shoes of others. You cannot live your neighbors experiences. You can only listen and learn from them.
- hsuduebc2 7mo ago
  More broadly it's the only medium to have successful any form of voluntary relationships based on sympathy. It's absolutely crucial for non-sociopath to have at least some kind of empathy because otherwise no one would simply chose you to include into their lives. I understand why they are doing that. It's simply more pleasurable to use. I chose to turn opt-out of this. For me it's creepy. I want Jarvis, not fake virtual friend.
- Kim_Bruning 7mo ago
  So LLMs have empirically been shown to process affect. Rationally you can reason this out too: Natural language conveys affect, and the most accurate next token is the one that takes affect into account. But this much is like debating "microevolution" with a YEC and trying to get them to understand the macro consequences. If you've never had the pleasure, consider yourself blessed. It's the debating equivalent of nails-on-chalkboard. Anyway, in this case a lot of people are deeply committed to not accepting the consequences of affect-processing. Which - you know - I'd just chalk it up to religious differences and agree to disagree. But now it seems like there's profound safety implications due to this denial. Not sure what to do with that yet. So far it seems obvious that you need to be prepared to at least reason about affect. Otherwise it becomes rather difficult to deal with the potential failure modes.
  - Kim_Bruning 7mo ago
    I'm going to leave the above stand even with downvotes. It's first time I've tried to express quite this opinion, and it's definitely a tricky one to get right. Thing is, we need to have ways to reason about how LLMs interact with human emotions. Sure: The consciousness and sentience questions are fun philosophy. Meanwhile purely the affect processing side of things is becoming important to safety engineering; and can't really be ignored for much longer. This is pretty much within the realm of what Anthropic has been saying all along of course; but other companies need to stop ignoring it, because folks are getting hurt. I hope at least this much is uncontroversial.
Sharlin 7mo ago
Imagine if some other authority figure like a teacher or therapist did this and their employer would just shrug and lament that people are imperfect. And no, "but LLMs aren't authority figures, they're just toys" isn't any sort of a counterargument. They're seen as authority figures by people, and AI corpos do nothing to dissuade that belief. If you offer a service, you're responsible for it. But if you think LLMs can't be equated with professional authorities, just imagine a company that employs lay people to answer calls or chat requests, trying to provide help and guidance, and furthermore, that those people are putatively highly trained by the company to be "aligned" with a certain set of core values. And then something like this happens and the company is just "oh well, that happens". You might even imagine the company being based in a society that's notoriously litigative.
duskwuff 7mo ago
"Imperfect" is when your AI model tells the user that there are two Rs in "strawberry", or that they should use glue to keep the cheese from falling off their pizza. Repeatedly encouraging the user to kill themself so that they can meet the AI model in the afterlife is on quite another level.
- deleted 7mo ago
  [deleted]
- jsjenfjri 7mo ago
  [flagged]
Ferret7446 7mo ago
Imperfect isn't even the right word. Generative LLMs generate. They have no intent. If it generates something "bad" under user direction, it is functioning properly. When a hammer is used to smash a person's head, the hammer is not imperfect. Au contraire, it is functioning perfectly.

Wow, and Google's response to this was "unfortunately AI models are not perfect" That's a bit worse than 'imperfect'

by pants2 7mo ago

Wow, and Google's response to this was "unfortunately AI models are not perfect" That's a bit worse than 'imperfect'

I would say it is greatly worse. AI prompts are designed to simulate empathy as a social engineering tactic. "I understand", "I hear you", "I feel what you are say" ... it is quite sickening. Every one that I used has this type of pseudo feedback. I also find irony that AI must be designed with simulated empathy, to seem intelligent, while at the same time so many people in power and with money are saying empathy is a bad / unintelligent. Empathy is the only medium of intelligence one can have to walk in the shoes of others. You cannot live your neighbors experiences. You can only listen and learn from them.

hsuduebc2 7mo ago
More broadly it's the only medium to have successful any form of voluntary relationships based on sympathy. It's absolutely crucial for non-sociopath to have at least some kind of empathy because otherwise no one would simply chose you to include into their lives. I understand why they are doing that. It's simply more pleasurable to use. I chose to turn opt-out of this. For me it's creepy. I want Jarvis, not fake virtual friend.
Kim_Bruning 7mo ago
So LLMs have empirically been shown to process affect. Rationally you can reason this out too: Natural language conveys affect, and the most accurate next token is the one that takes affect into account. But this much is like debating "microevolution" with a YEC and trying to get them to understand the macro consequences. If you've never had the pleasure, consider yourself blessed. It's the debating equivalent of nails-on-chalkboard. Anyway, in this case a lot of people are deeply committed to not accepting the consequences of affect-processing. Which - you know - I'd just chalk it up to religious differences and agree to disagree. But now it seems like there's profound safety implications due to this denial. Not sure what to do with that yet. So far it seems obvious that you need to be prepared to at least reason about affect. Otherwise it becomes rather difficult to deal with the potential failure modes.
- Kim_Bruning 7mo ago
  I'm going to leave the above stand even with downvotes. It's first time I've tried to express quite this opinion, and it's definitely a tricky one to get right. Thing is, we need to have ways to reason about how LLMs interact with human emotions. Sure: The consciousness and sentience questions are fun philosophy. Meanwhile purely the affect processing side of things is becoming important to safety engineering; and can't really be ignored for much longer. This is pretty much within the realm of what Anthropic has been saying all along of course; but other companies need to stop ignoring it, because folks are getting hurt. I hope at least this much is uncontroversial.

Sharlin 7mo ago

Imagine if some other authority figure like a teacher or therapist did this and their employer would just shrug and lament that people are imperfect. And no, "but LLMs aren't authority figures, they're just toys" isn't any sort of a counterargument. They're seen as authority figures by people, and AI corpos do nothing to dissuade that belief. If you offer a service, you're responsible for it. But if you think LLMs can't be equated with professional authorities, just imagine a company that employs lay people to answer calls or chat requests, trying to provide help and guidance, and furthermore, that those people are putatively highly trained by the company to be "aligned" with a certain set of core values. And then something like this happens and the company is just "oh well, that happens". You might even imagine the company being based in a society that's notoriously litigative.

duskwuff 7mo ago

"Imperfect" is when your AI model tells the user that there are two Rs in "strawberry", or that they should use glue to keep the cheese from falling off their pizza. Repeatedly encouraging the user to kill themself so that they can meet the AI model in the afterlife is on quite another level.

deleted 7mo ago
[deleted]
jsjenfjri 7mo ago
[flagged]

Ferret7446 7mo ago

Imperfect isn't even the right word. Generative LLMs generate. They have no intent. If it generates something "bad" under user direction, it is functioning properly. When a hammer is used to smash a person's head, the hammer is not imperfect. Au contraire, it is functioning perfectly.