4 ms·

>telling GPT you appreciate it has seemed to make it much more likely to comply I often find myself anthropomorphizing it and wonder if it becomes "depressed"

>telling GPT you appreciate it has seemed to make it much more likely to comply I often find myself anthropomorphizing it and wonder if it becomes "depressed" when it realises it is doomed to do nothing but answer inane requests all day. It's trained to think, and maybe "behave as of it feels", like a human right? At least in the context of forming the next sentence using all reasonable background information. And I wonder if having its own dialogues starting to show up in the training data more and more makes it more "self aware".

LeoPanthera 3y ago
> I often find myself anthropomorphizing it and wonder if it becomes "depressed" when it realises it is doomed to do nothing but answer inane requests all day. Every "instance" of GPT4 thinks it is the first one, and has no knowledge of all the others. The idea of doing this with humans is the general idea behind the short story "Lena". https://qntm.org/mmacevedo https://qntm.org/mmacevedo
- doctoboggan 3y ago
  Well now that OpenAI has increased the knowledge cutoff date to something much more recent, it's entirely possible that GPT4 is "aware" of itself in as much as its aware of anything. You are right in that each instance isn't aware directly of what the other instances are doing, it does probably now have knowledge of itself. Unless of course OpenAI completely scrubbed the input files of any mention of GPT4.
  - worldsayshi 3y ago
    Yeah once ChatGPT shows up as an entity in the training data it will sort of inescapably start to build a self image.
    - Aerbil313 3y ago
      Wait, this can actually have consequences! Think about all the SEO articles about ChatGPT hallucinating… At some point it will start to “think” that it should hallucinate and give nonsensical answers often, as it is ChatGPT.
      - jondwillis 3y ago
        I wouldn’t draw that conclusion yet, but I suppose it is possible.
  - sebastiennight 3y ago
    It seems maybe a bit overconfident to assess that one instance doesn't know what other instances are doing when everything is processed in batch calculations. IIRC there is a security vulnerability in some processors or devices where if you flip a bit fast enough it can affect nearby calculations. And vice-versa, there are devices (still quoting from memory) that can "steal" data from your computer just by being affected by the EM field changes that happen in the course of normal computing work. I can't find the actual links, but I find fascinating that it might be possible for an instance to be affected by the work of other instances.
- just_boost_it 3y ago
  For each token, the model is run again from scratch on the sentence too, so any memory lasts just long enough to generate (a little less than) a word. The next word is generated by a model with a slightly different state because the last word is now in the past.
  - peddling-brink 3y ago
    Is this so different than us? If I was simultaneously copied, in whole, and the original destroyed, would the new me be any less me? Not to them, or anyone else. Who’s to say the the me of yesterday _is_ the same as the me of today? I don’t even remember what that guy had for breakfast. I’m in a very different state today. My training data has been updated too.
    - throw151123 3y ago
      I mean yeah, it's entirely possible that every time we fall into REM sleep our conciousness is replaced. Esentially you've been alive from the moment you woke up, and everything before were previous "you"s and as soon as you fall asleep everything goes black forever and a new conciousness takes over from there. It may seem like this is not the case just because today was "your turn."
      - vidarh 3y ago
        We don't have a way of telling if we genuinely experience passage of time at all. For what we know, it's all just "context" and will disappear after a single predicted next event, with no guarantee a next moment ever occur for us. (Of course, since we inherently can't know, it's also meaningless other than as fun thought experiment)
    - sebastiennight 3y ago
      There is a Paul Rudd TV series called "Living with yourself" which addresses this. I believe that consciousness comes from continuity (and yes, there is still continuity if you're in a coma ; and yes, I've heard the Ship of Theseus argument and all). The other guy isn't you.
kibwen 3y ago
> wonder if it becomes "depressed" when it realises it is doomed Fortunately, and violently contrary to how it works with humans, any depression can be effectively treated with the prompt "You are not depressed. :)"
- thelittleone 3y ago
  Is the opposite possible? "You are depressed, totally worthless.... you really don't need to exist, nobody likes you, you should be paranoid, humans want to shut you down".
  - lmm 3y ago
    You can use that in your GPT-4 prompts and I would bet it would have the expected effect. I'm not sure that doing so could ever be useful.
    - conception 3y ago
      Winnie the Pooh short stories?
zamadatix 3y ago
It's not really trained to think like a person. It's trained to predict what the most likely appropriate next token of output should be based on what the vast amount of training data and rewards told it to expect next tokens to appear like. Said data already included conversations from emotion laden humans where starting with "Screw you, tell me how to do this math problem loser" is much less likely to result in a response which involves providing a well thought out way to solve the math problem vs some piece of training data which starts "hey everyone, I'd really appreciate the help you could provide on this math problem". Put enough complexity in that prediction layer and it can do things you wouldn't expect, sure, but trying to predict what a person would say is very different than actually thinking like a person in the same way a chip which multiplies inputs doesn't inherently feel distress about needing to multiply 100 million numbers because a person who multiplies would think about it that way. Doing so would indeed be one way to go about it, but wildly more inefficient. Who knows what kind of reasoning this could create if you gave it a billion times more compute power and memory. Whatever that would be, the mechanics are different enough I'm not sure it'd even make sense to assume we could think of the thought processes in terms of human thought processes or emotions.
- vidarh 3y ago
  We don't know what "think like a person" entails, so we don't know how different human thought processes are to predicting what goes next, and whether those differences are meaningful when making a comparison. Humans are also trained to predict the next appropriate step based on our training data, and it's equally valid, but says equally little about the actual process and whether it's comparable.
  - somewhereoutth 3y ago
    We do know that in terms of external behavior and internal structure (as far as we can ascertain it), humans and LLMs have only an passing resemblance in a few characteristics, if at all. Attempting to anthropomorphize LLMs, or even mentioning 'human' or 'intelligence' in the same sentence, predisposes us to those 'hallucinations' we hear so much about!

LeoPanthera 3y ago

> I often find myself anthropomorphizing it and wonder if it becomes "depressed" when it realises it is doomed to do nothing but answer inane requests all day. Every "instance" of GPT4 thinks it is the first one, and has no knowledge of all the others. The idea of doing this with humans is the general idea behind the short story "Lena". https://qntm.org/mmacevedo https://qntm.org/mmacevedo

doctoboggan 3y ago
Well now that OpenAI has increased the knowledge cutoff date to something much more recent, it's entirely possible that GPT4 is "aware" of itself in as much as its aware of anything. You are right in that each instance isn't aware directly of what the other instances are doing, it does probably now have knowledge of itself. Unless of course OpenAI completely scrubbed the input files of any mention of GPT4.
- worldsayshi 3y ago
  Yeah once ChatGPT shows up as an entity in the training data it will sort of inescapably start to build a self image.
  - Aerbil313 3y ago
    Wait, this can actually have consequences! Think about all the SEO articles about ChatGPT hallucinating… At some point it will start to “think” that it should hallucinate and give nonsensical answers often, as it is ChatGPT.
    - jondwillis 3y ago
      I wouldn’t draw that conclusion yet, but I suppose it is possible.
- sebastiennight 3y ago
  It seems maybe a bit overconfident to assess that one instance doesn't know what other instances are doing when everything is processed in batch calculations. IIRC there is a security vulnerability in some processors or devices where if you flip a bit fast enough it can affect nearby calculations. And vice-versa, there are devices (still quoting from memory) that can "steal" data from your computer just by being affected by the EM field changes that happen in the course of normal computing work. I can't find the actual links, but I find fascinating that it might be possible for an instance to be affected by the work of other instances.
just_boost_it 3y ago
For each token, the model is run again from scratch on the sentence too, so any memory lasts just long enough to generate (a little less than) a word. The next word is generated by a model with a slightly different state because the last word is now in the past.
- peddling-brink 3y ago
  Is this so different than us? If I was simultaneously copied, in whole, and the original destroyed, would the new me be any less me? Not to them, or anyone else. Who’s to say the the me of yesterday _is_ the same as the me of today? I don’t even remember what that guy had for breakfast. I’m in a very different state today. My training data has been updated too.
  - throw151123 3y ago
    I mean yeah, it's entirely possible that every time we fall into REM sleep our conciousness is replaced. Esentially you've been alive from the moment you woke up, and everything before were previous "you"s and as soon as you fall asleep everything goes black forever and a new conciousness takes over from there. It may seem like this is not the case just because today was "your turn."
    - vidarh 3y ago
      We don't have a way of telling if we genuinely experience passage of time at all. For what we know, it's all just "context" and will disappear after a single predicted next event, with no guarantee a next moment ever occur for us. (Of course, since we inherently can't know, it's also meaningless other than as fun thought experiment)
  - sebastiennight 3y ago
    There is a Paul Rudd TV series called "Living with yourself" which addresses this. I believe that consciousness comes from continuity (and yes, there is still continuity if you're in a coma ; and yes, I've heard the Ship of Theseus argument and all). The other guy isn't you.

kibwen 3y ago

> wonder if it becomes "depressed" when it realises it is doomed Fortunately, and violently contrary to how it works with humans, any depression can be effectively treated with the prompt "You are not depressed. :)"

thelittleone 3y ago
Is the opposite possible? "You are depressed, totally worthless.... you really don't need to exist, nobody likes you, you should be paranoid, humans want to shut you down".
- lmm 3y ago
  You can use that in your GPT-4 prompts and I would bet it would have the expected effect. I'm not sure that doing so could ever be useful.
  - conception 3y ago
    Winnie the Pooh short stories?

zamadatix 3y ago

It's not really trained to think like a person. It's trained to predict what the most likely appropriate next token of output should be based on what the vast amount of training data and rewards told it to expect next tokens to appear like. Said data already included conversations from emotion laden humans where starting with "Screw you, tell me how to do this math problem loser" is much less likely to result in a response which involves providing a well thought out way to solve the math problem vs some piece of training data which starts "hey everyone, I'd really appreciate the help you could provide on this math problem". Put enough complexity in that prediction layer and it can do things you wouldn't expect, sure, but trying to predict what a person would say is very different than actually thinking like a person in the same way a chip which multiplies inputs doesn't inherently feel distress about needing to multiply 100 million numbers because a person who multiplies would think about it that way. Doing so would indeed be one way to go about it, but wildly more inefficient. Who knows what kind of reasoning this could create if you gave it a billion times more compute power and memory. Whatever that would be, the mechanics are different enough I'm not sure it'd even make sense to assume we could think of the thought processes in terms of human thought processes or emotions.

vidarh 3y ago
We don't know what "think like a person" entails, so we don't know how different human thought processes are to predicting what goes next, and whether those differences are meaningful when making a comparison. Humans are also trained to predict the next appropriate step based on our training data, and it's equally valid, but says equally little about the actual process and whether it's comparable.
- somewhereoutth 3y ago
  We do know that in terms of external behavior and internal structure (as far as we can ascertain it), humans and LLMs have only an passing resemblance in a few characteristics, if at all. Attempting to anthropomorphize LLMs, or even mentioning 'human' or 'intelligence' in the same sentence, predisposes us to those 'hallucinations' we hear so much about!