4 ms·

I would say it is greatly worse. AI prompts are designed to simulate empathy as a social engineering tactic. "I understand", "I hear you", "I feel what you ar

I would say it is greatly worse. AI prompts are designed to simulate empathy as a social engineering tactic. "I understand", "I hear you", "I feel what you are say" ... it is quite sickening. Every one that I used has this type of pseudo feedback. I also find irony that AI must be designed with simulated empathy, to seem intelligent, while at the same time so many people in power and with money are saying empathy is a bad / unintelligent. Empathy is the only medium of intelligence one can have to walk in the shoes of others. You cannot live your neighbors experiences. You can only listen and learn from them.

hsuduebc2 8mo ago
More broadly it's the only medium to have successful any form of voluntary relationships based on sympathy. It's absolutely crucial for non-sociopath to have at least some kind of empathy because otherwise no one would simply chose you to include into their lives. I understand why they are doing that. It's simply more pleasurable to use. I chose to turn opt-out of this. For me it's creepy. I want Jarvis, not fake virtual friend.
Kim_Bruning 8mo ago
So LLMs have empirically been shown to process affect. Rationally you can reason this out too: Natural language conveys affect, and the most accurate next token is the one that takes affect into account. But this much is like debating "microevolution" with a YEC and trying to get them to understand the macro consequences. If you've never had the pleasure, consider yourself blessed. It's the debating equivalent of nails-on-chalkboard. Anyway, in this case a lot of people are deeply committed to not accepting the consequences of affect-processing. Which - you know - I'd just chalk it up to religious differences and agree to disagree. But now it seems like there's profound safety implications due to this denial. Not sure what to do with that yet. So far it seems obvious that you need to be prepared to at least reason about affect. Otherwise it becomes rather difficult to deal with the potential failure modes.
- Kim_Bruning 8mo ago
  I'm going to leave the above stand even with downvotes. It's first time I've tried to express quite this opinion, and it's definitely a tricky one to get right. Thing is, we need to have ways to reason about how LLMs interact with human emotions. Sure: The consciousness and sentience questions are fun philosophy. Meanwhile purely the affect processing side of things is becoming important to safety engineering; and can't really be ignored for much longer. This is pretty much within the realm of what Anthropic has been saying all along of course; but other companies need to stop ignoring it, because folks are getting hurt. I hope at least this much is uncontroversial.

hsuduebc2 8mo ago

More broadly it's the only medium to have successful any form of voluntary relationships based on sympathy. It's absolutely crucial for non-sociopath to have at least some kind of empathy because otherwise no one would simply chose you to include into their lives. I understand why they are doing that. It's simply more pleasurable to use. I chose to turn opt-out of this. For me it's creepy. I want Jarvis, not fake virtual friend.

Kim_Bruning 8mo ago

So LLMs have empirically been shown to process affect. Rationally you can reason this out too: Natural language conveys affect, and the most accurate next token is the one that takes affect into account. But this much is like debating "microevolution" with a YEC and trying to get them to understand the macro consequences. If you've never had the pleasure, consider yourself blessed. It's the debating equivalent of nails-on-chalkboard. Anyway, in this case a lot of people are deeply committed to not accepting the consequences of affect-processing. Which - you know - I'd just chalk it up to religious differences and agree to disagree. But now it seems like there's profound safety implications due to this denial. Not sure what to do with that yet. So far it seems obvious that you need to be prepared to at least reason about affect. Otherwise it becomes rather difficult to deal with the potential failure modes.

Kim_Bruning 8mo ago
I'm going to leave the above stand even with downvotes. It's first time I've tried to express quite this opinion, and it's definitely a tricky one to get right. Thing is, we need to have ways to reason about how LLMs interact with human emotions. Sure: The consciousness and sentience questions are fun philosophy. Meanwhile purely the affect processing side of things is becoming important to safety engineering; and can't really be ignored for much longer. This is pretty much within the realm of what Anthropic has been saying all along of course; but other companies need to stop ignoring it, because folks are getting hurt. I hope at least this much is uncontroversial.