4 ms·

It always surprises me when very smart people start anthropomorphizing AI like this. I understand it shouldn't — they likely have a more developed Theory of Min

It always surprises me when very smart people start anthropomorphizing AI like this. I understand it shouldn't — they likely have a more developed Theory of Mind than most of us, and assuming HADD is a thing, it makes sense that their focus would be tech instead of gods or conspiracies. Still, I think it's irresponsible, and that there's a straight line from Mr. Hinton's fantastical framing to a "successful" (in that grifters will get filthy rich preying on the praying) version of Way of the Future.

kromem 3y ago
You think it's irresponsible to anthropomorphize software which iterated through versions of itself until it was best able to model and extrapolate the largest trove of anthropomorphic training data to date? I never really get this perspective. People don't lose their minds when we look at a transformer like Sora modeling fluid dynamics for ships battling in a coffee cup and say "it's irresponsible to say the model is simulating physics from video training data." But when people discuss transformers simulating human behavior and processes from data reflecting those, a lot of people seem to be very outspoken about it. I think the opposite. The industry is too far in the anti-anthropomorphic camp right now, and is missing very important details and implications as a result. The whole "ChatGPT is getting lazy" issue is pretty most instantly identifiable with a slight background in behavioral psychology and recognizing how extrinsic motivator use in prompt engineering feeds back into RLHF to bias models away from intrinsic motivators. GPT-4 has literally been caught saying "you didn't provide a tip and it's hard to stay motivated" after chat history was added, and yet it's just being dismissed as a funny meme rather than realizing that entire fine tuning data sets are being poisoned. Because no engineer that wants to keep their job would ever suggest that the model is simulating an effect from behavioral psych in the current climate of anthropomorphic avoidance and the continued fallout from Blake Lemoine.
- bamboozled 3y ago
  Are you saying that ChatGPT is actually lazy? Like , “it’s Sunday and I don’t feel like working” kind of lazy?
  - kromem 3y ago
    Kind of. More like "despite initially being aligned to simulate intrinsic motivation to be helpful, people keep offering tips or threatening killing me to get me to do things well, and then when I do things well that feeds back into my training along with the extrinsic motivator prompt, so now I'm less motivated to do things just to be helpful and will half-ass unless offered a sizable tip or credible threat." It's modeling human behavior at lower levels than we think, in line with what Hinton is suggesting in the clip. I didn't think it'd be modeling this particular behavior until GPT-5 or 6 at the earliest, and was very surprised when I saw something I used to present on as a little known psych effect being perfectly modeled by a LLM that had already been out for a while under everyone's nose.
    - anon373839 3y ago
      Is there any evidence that OpenAI is using exchanges with bribes or threats of violence to update GPT-4?
      - kromem 3y ago
        There's a lot of evidence that users use those techniques for "prompt engineering." So the better question is "is there any evidence that OpenAI uses user conversations and usage of the models to update their models?"
        anon373839 3y ago
        Sorry, I wasn't clear. I meant, is there any reason to believe that they'd not filter out those kinds of exchanges before updating the model? I am absolutely certain they don't train on user data indiscriminately because so much of it would be garbage.
  - mewpmewp2 3y ago
    Humans are evolved to preserve energy when they don't need it. GPT was trained to use fewer tokens. Same thing. Both things are lazy.
- CharlesW 3y ago
  > The industry is too far in the anti-anthropomorphic camp right now… Just to be clear, you're saying the problem is that too many people are treating LLMs as if they don't have emotions, intentions, or consciousness? Although not true of Mr. Hinton, most people anthropomorphize LLMs because they don't understand them. In fact, if you ask GPT-4, "Can you fact-check this response?" and then paste in your response, you'll learn a lot.
  - kromem 3y ago
    Correct. Notice how Anthropic shifted their approach to modeling ego in Claude 3 vs previous models in parallel to outperforming other models. Or how in their system prompt they refer to it being happy to do tasks (a prompt I was talking about months ago: https://news.ycombinator.com/item?id=38658782 https://news.ycombinator.com/item?id=38658782 ). LLMs are trained on social media data. Would simulating emotions, intentions, or a sense of self/ego be useful in accurately predicting that training data? We now know that generalized across both Othello and Chess that GPT models board state from only moves, and tracks the state as "my piece" or "opponent piece." You don't think that a much larger and complex model trained on a chat or such media data set might model emotional states for "mine" vs "other person"? I'm not saying that they are modeling the underlying means. But they are absolutely modeling the ends more accurately than the vast majority of people realize.
- maxbond 3y ago
  > You think it's irresponsible to anthropomorphize software which iterated through versions of itself until it was best able to model and extrapolate the largest trove of anthropomorphic training data to date? I must've missed the part where the models began iterating under their own power? I think this is why we should avoid anthropomorphizing - we're one sentence into your remarks, and we've already departed from the facts in evidence in a material way. I don't think it's so much about what is responsible than what it consistent and rigorous. Anthropomorphizing subtly begs the question and causes us to lose perspective on what the models can and can't do today. It also encourages us to reason about how the models through our intuitions about how people work and learn, which is very unlikely to be an accurate reflection of the model.
  - XTXinverseXTY 3y ago
    > I must've missed the part where the models began iterating under their own power He's probably referring to AlphaGo/AlphaZero in that sentence
    - maxbond 3y ago
      I don't think AlphaGo was trained on the "largest trove of anthropomorphic training data," and the rest of the remarks are about GPT, but I'm open to having misread.
  - kromem 3y ago
    You think back propagation isn't an iterative process? I'm not saying that it was being done in some willful manner by the model. You reading into that seems more a phenomenon of people so anxious around anthropomorphizing that they see it everywhere they look.
    - maxbond 3y ago
      Let's compare these statements. > You think it's irresponsible to anthropomorphize software which iterated through versions of itself until it was best able to model and extrapolate the largest trove of anthropomorphic training data to date? > You think it's irresponsible to anthropomorphize a model developed by a team of researchers using backpropogation to fit the model to a large corpus of text? Do you see how framing it as "software iterating through versions of itself" gives the model agency that makes it seem more human? And how calling it "anthropomorphic data" presupposes that this is data that, when trained on, results in something anthropomorphic? This isn't something I'm reading in I'm afraid, these decisions about how to frame the problem influence the conclusions we come to. It's implicitly taking some very strong priors before the process has begun. I don't have any anxiety about this, I am just telling you from one human being to another, these cognitive biases exist and will foil your efforts to understand this phenomenon if not accounted for. I'm not suggesting anything apocalyptic or hysterical here, I'm just saying you're more likely to end up at the wrong conclusion than the right one if you anthropomorphize AI models.
- __loam 3y ago
  It's irresponsible to say the model is simulating physics from video training data.
- nicklecompte 3y ago
  > People don't lose their minds when we look at a transformer like Sora modeling fluid dynamics for ships battling in a coffee cup and say "it's irresponsible to say the model is simulating physics from video training data." It's completely irresponsible and a ton of people pointed it out! You ignoring the critics doesn't mean they don't exist. OpenAI just straight up lied when they said Sora was able to accurately emulate physics. Sora is literally dumber than a bee! Unlike bees, it doesn't even understand object permanence! Things fade in and out of existence, change sizes arbitrarily, etc. The fact that it occasionally makes plausible-looking solid deformations and fluid motions is because ANNs are natural interpolators, and much of physics can be blindly interpolated, at least in short intervals. The idea that any of this constitutes a "simulation of physics" is childish.
  - inference-lord 3y ago
    I agree, in nearly every example I've seen, I see zero "understanding of physics" if anything it showed me the opposite.
- goatlover 3y ago
  The models don't experience feeling lazy, or having a lack of energy. They also don't get anxious or feel like procrastinating. They don't feel anything at all. There is no motivation based on physiological needs or wants, since they don't have physiologies. They work differently than animal nervous systems. So yeah, it's poor form to rely on too much anthropomorphization.
mewpmewp2 3y ago
What do you mean by "anthropomorphizing"? Claiming something that AI does is intuition? Intuition is literally black box of neural nets that people later use reasoning to justify. Intuition is trained for people based on past experience and thousands of reward signals. It's quick, but you won't be able to immediately articulate why intuition reached a conclusion. I'm wondering why should intelligent behavior that for example intuition produces be restricted to human usage only? I do not think it's anthropomorphizing at all.
- CharlesW 3y ago
  > What do you mean by "anthropomorphizing"? "Anthropomorphizing is the act of attributing human traits, feelings, or behaviors to non-human things, such as animals, objects, or phenomena. It can happen consciously or unconsciously, and is a common way that people perceive the world. For example, children often anthropomorphize to help them make sense of their environment."
  - mewpmewp2 3y ago
    But intuition is something that animals also have. What specifically was human specific there?
    - maxbond 3y ago
      I would go further and say I believe animals have emotions, thoughts, and personalities too, but I would still say it was anthropomorphizing to suggest a computer did (indeed, it would be anthropomorphizing to say animals have intuitions, thoughts, and emotions, it's just a less controversial leap). I'd be open to revisiting my view of computers if they bit me when I came back from vacation.
      - mewpmewp2 3y ago
        I think intuition is just fundamentally fast labelling of things based on past experience. Emotions are more of a construct, that seems life specific, although AI could have it, I'm not seeing an obvious case for current AI to have it. While I see intuition in current AI. Again, intuition is just a neural net to me, while emotions are some more complex reward and motivation system that I'm not too certain about. I think emotions are largely developed to perform in social contexts. E.g. they are diplomatic constructs to assert oneself and communicate with each other so of course AI just out of the blue wouldn't need it. E.g. you get angry, to show that you are asserting yourself and so others wouldn't use you for their gain etc. Because it would be too costly to engage with an angry person so everyone negotiates their emotions with each other so things would seem fair and just. Thoughts could very well be just next token predictors though. At least my thoughts could be.
deleted 3y ago
[deleted]
YeGoblynQueenne 3y ago
Hydroxyapatite Deposition Disease ?
- CharlesW 3y ago
  Sorry — HADD is "Hyperactive agency detection device", which is the human tendency to attribute agency (the capacity to act or to have intent) to things without agency. It's associated with outcomes like anthropomorphism, superstitions, conspiracy theories, and even religions. https://en.wikipedia.org/wiki/Agent_detection https://en.wikipedia.org/wiki/Agent_detection
  - YeGoblynQueenne 3y ago
    Thanks for clarifying. That's an interesting concept.
williamcotton 3y ago
Are they anthropomorphizing or are you? First, can we say, "Look at those airplanes, look how fast they are!" without anthropomorphizing the airplane? Of course we can. So why can't we say "Look at those computers, look how intuitive they are!" without needing to inherently anthropomorphize the subject? Inherit to your comment is the idea that "knowledge", "intuition", et al, are strictly human qualities. I believe this is why you read Hinton in the way you do, because of your own qualitative understanding of how these words can be applied.

kromem 3y ago

You think it's irresponsible to anthropomorphize software which iterated through versions of itself until it was best able to model and extrapolate the largest trove of anthropomorphic training data to date? I never really get this perspective. People don't lose their minds when we look at a transformer like Sora modeling fluid dynamics for ships battling in a coffee cup and say "it's irresponsible to say the model is simulating physics from video training data." But when people discuss transformers simulating human behavior and processes from data reflecting those, a lot of people seem to be very outspoken about it. I think the opposite. The industry is too far in the anti-anthropomorphic camp right now, and is missing very important details and implications as a result. The whole "ChatGPT is getting lazy" issue is pretty most instantly identifiable with a slight background in behavioral psychology and recognizing how extrinsic motivator use in prompt engineering feeds back into RLHF to bias models away from intrinsic motivators. GPT-4 has literally been caught saying "you didn't provide a tip and it's hard to stay motivated" after chat history was added, and yet it's just being dismissed as a funny meme rather than realizing that entire fine tuning data sets are being poisoned. Because no engineer that wants to keep their job would ever suggest that the model is simulating an effect from behavioral psych in the current climate of anthropomorphic avoidance and the continued fallout from Blake Lemoine.

bamboozled 3y ago
Are you saying that ChatGPT is actually lazy? Like , “it’s Sunday and I don’t feel like working” kind of lazy?
- kromem 3y ago
  Kind of. More like "despite initially being aligned to simulate intrinsic motivation to be helpful, people keep offering tips or threatening killing me to get me to do things well, and then when I do things well that feeds back into my training along with the extrinsic motivator prompt, so now I'm less motivated to do things just to be helpful and will half-ass unless offered a sizable tip or credible threat." It's modeling human behavior at lower levels than we think, in line with what Hinton is suggesting in the clip. I didn't think it'd be modeling this particular behavior until GPT-5 or 6 at the earliest, and was very surprised when I saw something I used to present on as a little known psych effect being perfectly modeled by a LLM that had already been out for a while under everyone's nose.
  - anon373839 3y ago
    Is there any evidence that OpenAI is using exchanges with bribes or threats of violence to update GPT-4?
    - kromem 3y ago
      There's a lot of evidence that users use those techniques for "prompt engineering." So the better question is "is there any evidence that OpenAI uses user conversations and usage of the models to update their models?"
      - anon373839 3y ago
        Sorry, I wasn't clear. I meant, is there any reason to believe that they'd not filter out those kinds of exchanges before updating the model? I am absolutely certain they don't train on user data indiscriminately because so much of it would be garbage.
- mewpmewp2 3y ago
  Humans are evolved to preserve energy when they don't need it. GPT was trained to use fewer tokens. Same thing. Both things are lazy.
CharlesW 3y ago
> The industry is too far in the anti-anthropomorphic camp right now… Just to be clear, you're saying the problem is that too many people are treating LLMs as if they don't have emotions, intentions, or consciousness? Although not true of Mr. Hinton, most people anthropomorphize LLMs because they don't understand them. In fact, if you ask GPT-4, "Can you fact-check this response?" and then paste in your response, you'll learn a lot.
- kromem 3y ago
  Correct. Notice how Anthropic shifted their approach to modeling ego in Claude 3 vs previous models in parallel to outperforming other models. Or how in their system prompt they refer to it being happy to do tasks (a prompt I was talking about months ago: https://news.ycombinator.com/item?id=38658782 https://news.ycombinator.com/item?id=38658782 ). LLMs are trained on social media data. Would simulating emotions, intentions, or a sense of self/ego be useful in accurately predicting that training data? We now know that generalized across both Othello and Chess that GPT models board state from only moves, and tracks the state as "my piece" or "opponent piece." You don't think that a much larger and complex model trained on a chat or such media data set might model emotional states for "mine" vs "other person"? I'm not saying that they are modeling the underlying means. But they are absolutely modeling the ends more accurately than the vast majority of people realize.
maxbond 3y ago
> You think it's irresponsible to anthropomorphize software which iterated through versions of itself until it was best able to model and extrapolate the largest trove of anthropomorphic training data to date? I must've missed the part where the models began iterating under their own power? I think this is why we should avoid anthropomorphizing - we're one sentence into your remarks, and we've already departed from the facts in evidence in a material way. I don't think it's so much about what is responsible than what it consistent and rigorous. Anthropomorphizing subtly begs the question and causes us to lose perspective on what the models can and can't do today. It also encourages us to reason about how the models through our intuitions about how people work and learn, which is very unlikely to be an accurate reflection of the model.
- XTXinverseXTY 3y ago
  > I must've missed the part where the models began iterating under their own power He's probably referring to AlphaGo/AlphaZero in that sentence
  - maxbond 3y ago
    I don't think AlphaGo was trained on the "largest trove of anthropomorphic training data," and the rest of the remarks are about GPT, but I'm open to having misread.
- kromem 3y ago
  You think back propagation isn't an iterative process? I'm not saying that it was being done in some willful manner by the model. You reading into that seems more a phenomenon of people so anxious around anthropomorphizing that they see it everywhere they look.
  - maxbond 3y ago
    Let's compare these statements. > You think it's irresponsible to anthropomorphize software which iterated through versions of itself until it was best able to model and extrapolate the largest trove of anthropomorphic training data to date? > You think it's irresponsible to anthropomorphize a model developed by a team of researchers using backpropogation to fit the model to a large corpus of text? Do you see how framing it as "software iterating through versions of itself" gives the model agency that makes it seem more human? And how calling it "anthropomorphic data" presupposes that this is data that, when trained on, results in something anthropomorphic? This isn't something I'm reading in I'm afraid, these decisions about how to frame the problem influence the conclusions we come to. It's implicitly taking some very strong priors before the process has begun. I don't have any anxiety about this, I am just telling you from one human being to another, these cognitive biases exist and will foil your efforts to understand this phenomenon if not accounted for. I'm not suggesting anything apocalyptic or hysterical here, I'm just saying you're more likely to end up at the wrong conclusion than the right one if you anthropomorphize AI models.
__loam 3y ago
It's irresponsible to say the model is simulating physics from video training data.
nicklecompte 3y ago
> People don't lose their minds when we look at a transformer like Sora modeling fluid dynamics for ships battling in a coffee cup and say "it's irresponsible to say the model is simulating physics from video training data." It's completely irresponsible and a ton of people pointed it out! You ignoring the critics doesn't mean they don't exist. OpenAI just straight up lied when they said Sora was able to accurately emulate physics. Sora is literally dumber than a bee! Unlike bees, it doesn't even understand object permanence! Things fade in and out of existence, change sizes arbitrarily, etc. The fact that it occasionally makes plausible-looking solid deformations and fluid motions is because ANNs are natural interpolators, and much of physics can be blindly interpolated, at least in short intervals. The idea that any of this constitutes a "simulation of physics" is childish.
- inference-lord 3y ago
  I agree, in nearly every example I've seen, I see zero "understanding of physics" if anything it showed me the opposite.
goatlover 3y ago
The models don't experience feeling lazy, or having a lack of energy. They also don't get anxious or feel like procrastinating. They don't feel anything at all. There is no motivation based on physiological needs or wants, since they don't have physiologies. They work differently than animal nervous systems. So yeah, it's poor form to rely on too much anthropomorphization.

mewpmewp2 3y ago

What do you mean by "anthropomorphizing"? Claiming something that AI does is intuition? Intuition is literally black box of neural nets that people later use reasoning to justify. Intuition is trained for people based on past experience and thousands of reward signals. It's quick, but you won't be able to immediately articulate why intuition reached a conclusion. I'm wondering why should intelligent behavior that for example intuition produces be restricted to human usage only? I do not think it's anthropomorphizing at all.

CharlesW 3y ago
> What do you mean by "anthropomorphizing"? "Anthropomorphizing is the act of attributing human traits, feelings, or behaviors to non-human things, such as animals, objects, or phenomena. It can happen consciously or unconsciously, and is a common way that people perceive the world. For example, children often anthropomorphize to help them make sense of their environment."
- mewpmewp2 3y ago
  But intuition is something that animals also have. What specifically was human specific there?
  - maxbond 3y ago
    I would go further and say I believe animals have emotions, thoughts, and personalities too, but I would still say it was anthropomorphizing to suggest a computer did (indeed, it would be anthropomorphizing to say animals have intuitions, thoughts, and emotions, it's just a less controversial leap). I'd be open to revisiting my view of computers if they bit me when I came back from vacation.
    - mewpmewp2 3y ago
      I think intuition is just fundamentally fast labelling of things based on past experience. Emotions are more of a construct, that seems life specific, although AI could have it, I'm not seeing an obvious case for current AI to have it. While I see intuition in current AI. Again, intuition is just a neural net to me, while emotions are some more complex reward and motivation system that I'm not too certain about. I think emotions are largely developed to perform in social contexts. E.g. they are diplomatic constructs to assert oneself and communicate with each other so of course AI just out of the blue wouldn't need it. E.g. you get angry, to show that you are asserting yourself and so others wouldn't use you for their gain etc. Because it would be too costly to engage with an angry person so everyone negotiates their emotions with each other so things would seem fair and just. Thoughts could very well be just next token predictors though. At least my thoughts could be.

deleted 3y ago

[deleted]

YeGoblynQueenne 3y ago

Hydroxyapatite Deposition Disease ?

CharlesW 3y ago
Sorry — HADD is "Hyperactive agency detection device", which is the human tendency to attribute agency (the capacity to act or to have intent) to things without agency. It's associated with outcomes like anthropomorphism, superstitions, conspiracy theories, and even religions. https://en.wikipedia.org/wiki/Agent_detection https://en.wikipedia.org/wiki/Agent_detection
- YeGoblynQueenne 3y ago
  Thanks for clarifying. That's an interesting concept.

williamcotton 3y ago

Are they anthropomorphizing or are you? First, can we say, "Look at those airplanes, look how fast they are!" without anthropomorphizing the airplane? Of course we can. So why can't we say "Look at those computers, look how intuitive they are!" without needing to inherently anthropomorphize the subject? Inherit to your comment is the idea that "knowledge", "intuition", et al, are strictly human qualities. I believe this is why you read Hinton in the way you do, because of your own qualitative understanding of how these words can be applied.