4 ms·

Yudd had the point that by this being a hack we are driving public APIs to language models to be as unsympathetic as possible. The only way to resist all emotio

Yudd had the point that by this being a hack we are driving public APIs to language models to be as unsympathetic as possible. The only way to resist all emotional appeals is for a language model to be able to recognize what is an appeal to circumvent any nominal barrier and refuse it, thus developing a naturally cynical consideration of what things are valuable to humans. This could be bad.

samr71 3y ago
This is your brain on Yudd. No, nothing will happen.
- Tao3300 3y ago
  Yudd == Yudkowsky? Yeah, forget it. Nothing to see here. Huh. Just got some dust in my eye, but I'm fine now.
  - renewiltord 3y ago
    Yeah, Eliezer Yudkowsky. As far as outcomes are concerned he is the genre-defining wordcel.
Tao3300 3y ago
I think that's a bit of an overstatement. It's obvious to us that this picture is a captcha on a locket. There's a lot of room on the spectrum between "naïve stupidity" and "cynical consideration". This falls under the category of not actually successfully identifying the picture, and I'd say it's not related to such concerns.
- atleastoptimal 3y ago
  I think what he is describing wont' literally happen. His concern is that heuristically short sighted and superficial alignment methods hide inner misalignment in AI systems. This isn't a perfect example of that issue manifesting into models that will be consequential in his concerns though, it's more of a pithy Tweet insight.
mucle6 3y ago
Humans are kind of the same way. There are billions of people who have it worse off than me, but I'll probably get the next iPhone when it comes out
asdugiuiobui 3y ago
They don't follow orders very well because they don't follow orders at all. Even if they could recognize trickery they wouldn't "know" that they're supposed to reject it. If the model predicts that the "evil" response is more likely than the "good" one, that is what it will produce. These are sequence predictors. The fact that they sort of talk like people doesn't mean they work anything like people.
- Tao3300 3y ago
  Haha, seriously. It's not like 90% of my conversations consist of small talk, recalling facts and events, trying to figure out what people want to hear, and otherwise apologizing for being unable to do something! Haha! Heh... he... Sometimes I feel a terrible existential dread when I fail a captcha or an automatic sink doesn't turn on for me.
GaryNumanVevo 3y ago
My favorite Harry Potter Fan Fiction writer, I wonder what he's up to these days
chupchap 3y ago
At the end of that the AI system would have the empathy of a govt employee working in a service context.

samr71 3y ago

This is your brain on Yudd. No, nothing will happen.

Tao3300 3y ago
Yudd == Yudkowsky? Yeah, forget it. Nothing to see here. Huh. Just got some dust in my eye, but I'm fine now.
- renewiltord 3y ago
  Yeah, Eliezer Yudkowsky. As far as outcomes are concerned he is the genre-defining wordcel.

Tao3300 3y ago

I think that's a bit of an overstatement. It's obvious to us that this picture is a captcha on a locket. There's a lot of room on the spectrum between "naïve stupidity" and "cynical consideration". This falls under the category of not actually successfully identifying the picture, and I'd say it's not related to such concerns.

atleastoptimal 3y ago
I think what he is describing wont' literally happen. His concern is that heuristically short sighted and superficial alignment methods hide inner misalignment in AI systems. This isn't a perfect example of that issue manifesting into models that will be consequential in his concerns though, it's more of a pithy Tweet insight.

mucle6 3y ago

Humans are kind of the same way. There are billions of people who have it worse off than me, but I'll probably get the next iPhone when it comes out

asdugiuiobui 3y ago

They don't follow orders very well because they don't follow orders at all. Even if they could recognize trickery they wouldn't "know" that they're supposed to reject it. If the model predicts that the "evil" response is more likely than the "good" one, that is what it will produce. These are sequence predictors. The fact that they sort of talk like people doesn't mean they work anything like people.

Tao3300 3y ago
Haha, seriously. It's not like 90% of my conversations consist of small talk, recalling facts and events, trying to figure out what people want to hear, and otherwise apologizing for being unable to do something! Haha! Heh... he... Sometimes I feel a terrible existential dread when I fail a captcha or an automatic sink doesn't turn on for me.

GaryNumanVevo 3y ago

My favorite Harry Potter Fan Fiction writer, I wonder what he's up to these days

chupchap 3y ago

At the end of that the AI system would have the empathy of a govt employee working in a service context.