3 ms·

Are you saying that ChatGPT is actually lazy? Like , “it’s Sunday and I don’t feel like working” kind of lazy?

kromem 3y ago
Kind of. More like "despite initially being aligned to simulate intrinsic motivation to be helpful, people keep offering tips or threatening killing me to get me to do things well, and then when I do things well that feeds back into my training along with the extrinsic motivator prompt, so now I'm less motivated to do things just to be helpful and will half-ass unless offered a sizable tip or credible threat." It's modeling human behavior at lower levels than we think, in line with what Hinton is suggesting in the clip. I didn't think it'd be modeling this particular behavior until GPT-5 or 6 at the earliest, and was very surprised when I saw something I used to present on as a little known psych effect being perfectly modeled by a LLM that had already been out for a while under everyone's nose.
- anon373839 3y ago
  Is there any evidence that OpenAI is using exchanges with bribes or threats of violence to update GPT-4?
  - kromem 3y ago
    There's a lot of evidence that users use those techniques for "prompt engineering." So the better question is "is there any evidence that OpenAI uses user conversations and usage of the models to update their models?"
    - anon373839 3y ago
      Sorry, I wasn't clear. I meant, is there any reason to believe that they'd not filter out those kinds of exchanges before updating the model? I am absolutely certain they don't train on user data indiscriminately because so much of it would be garbage.
mewpmewp2 3y ago
Humans are evolved to preserve energy when they don't need it. GPT was trained to use fewer tokens. Same thing. Both things are lazy.
- bamboozled 3y ago
  My car is aerodynamically designed to use less fuel, are you going to claim this makes my car lazy? Should I go talk to my car and ask it if it's being lazy?
  - mewpmewp2 3y ago
    Aerodynamics would not be enough. That is comparable to more lighter person or a more lower or optimised parameter count LLM though. Laziness requires some sort of reward mechanism. Human is lazy when an action doesn't seem to be worth the reward.
    - bamboozled 3y ago
      Tuning a system to be lazy is different to a person feeling like they want to be lazy is what I'm saying.
      - mewpmewp2 3y ago
        But people were tuned to be lazy by natural selection and evolution.
- goatlover 3y ago
  No they're not. That's just misusing language. Efficiency isn't the same thing as laziness.

Kind of. More like "despite initially being aligned to simulate intrinsic motivation to be helpful, people keep offering tips or threatening killing me to get me to do things well, and then when I do things well that feeds back into my training along with the extrinsic motivator prompt, so now I'm less motivated to do things just to be helpful and will half-ass unless offered a sizable tip or credible threat." It's modeling human behavior at lower levels than we think, in line with what Hinton is suggesting in the clip. I didn't think it'd be modeling this particular behavior until GPT-5 or 6 at the earliest, and was very surprised when I saw something I used to present on as a little known psych effect being perfectly modeled by a LLM that had already been out for a while under everyone's nose.

anon373839 3y ago
Is there any evidence that OpenAI is using exchanges with bribes or threats of violence to update GPT-4?
- kromem 3y ago
  There's a lot of evidence that users use those techniques for "prompt engineering." So the better question is "is there any evidence that OpenAI uses user conversations and usage of the models to update their models?"
  - anon373839 3y ago
    Sorry, I wasn't clear. I meant, is there any reason to believe that they'd not filter out those kinds of exchanges before updating the model? I am absolutely certain they don't train on user data indiscriminately because so much of it would be garbage.

mewpmewp2 3y ago

Humans are evolved to preserve energy when they don't need it. GPT was trained to use fewer tokens. Same thing. Both things are lazy.

bamboozled 3y ago
My car is aerodynamically designed to use less fuel, are you going to claim this makes my car lazy? Should I go talk to my car and ask it if it's being lazy?
- mewpmewp2 3y ago
  Aerodynamics would not be enough. That is comparable to more lighter person or a more lower or optimised parameter count LLM though. Laziness requires some sort of reward mechanism. Human is lazy when an action doesn't seem to be worth the reward.
  - bamboozled 3y ago
    Tuning a system to be lazy is different to a person feeling like they want to be lazy is what I'm saying.
    - mewpmewp2 3y ago
      But people were tuned to be lazy by natural selection and evolution.
goatlover 3y ago
No they're not. That's just misusing language. Efficiency isn't the same thing as laziness.