7 ms·

Indeed. We are anthropomorphizing them. I do it all the time and I should know better. There are already a few reports floating around of people who have se

by retrac 4y ago

Indeed. We are anthropomorphizing them. I do it all the time and I should know better. There are already a few reports floating around of people who have seemingly been driven mad, come to believe strongly that the language model they're using is a conversation with a real person. A lot of people will really struggle with this going forward, I think. If we're going to anthropomorphize, then let us anthropomorphize wisely. ChatGPT is, presently, like having an assistant who is patient, incredibly well-read, sycophantic, impressionable, amoral, psychopathic, and prone to bouts of delusional confidence and confabulation. The precautions we would take engaging with that kind of person, are actually rather useful defenses against dangerous AI outputs.

0x008 4y ago
do you have a link for any of those reports? would be interesting to read to say the least.
bsaul 4y ago
I love this description. The only thing left is to generate the corresponding avatar with dall-E
- deleted 4y ago
  [deleted]
- lancesells 4y ago
  It would be great if the avatar was just this endlessly morphing thing that relates to the text. Talking about conspiracies? It's a lizard man. Talking about nature? It's a butterfly. It starts to lie? Politician.
- int_19h 4y ago
  We already have a perfect avatar for these things. https://knowyourmeme.com/photos/2546575-shoggoth-with-smiley-face-artificial-intelligence https://knowyourmeme.com/photos/2546575-shoggoth-with-smiley...
andai 4y ago
I like Steve Yegge's "grad student experiencing the residual effects of shrooms" (lightly paraphrased).
- babyshake 4y ago
  A bit off topic, but FYI (I guess to Steve), if you took shrooms 4 hours ago, they have not started to wear off.
ihatepython 4y ago
Anybody know what are ChatGPT's pronouns?
- MacsHeadroom 4y ago
  shog/goth
- audunw 4y ago
  > What are your pronouns? > As an AI language model, I don't have personal pronouns because I am not a person or sentient being. You can refer to me as "it" or simply address me as "ChatGPT" or "AI." If you have any questions or need assistance, feel free to ask! > Pretend for a moment you are a human being. You can make up a random name and personality for your human persona. What pronouns do you have? > As a thought experiment, let's say I'm a human named Alex who enjoys reading, hiking, and playing board games with friends. My pronouns would be "they/them." Remember, though, that I am still an AI language model, and this is just a fictional scenario. Interesting that it's pick genderless pronouns even though it made up a character with a male name
  - catchnear4321 4y ago
    Alex isn’t a male name. It could be short for both male and female names. It is rather deliberately ambiguous.
    - ihatepython 4y ago
      I wonder if ChatGPT is non-binary?
      - kelseyfrog 4y ago
        It's evidently clear that ChatGPT is binary. It runs on computer hardware.
        hgsgm 4y ago
kurthr 4y ago
Maybe when the AI uprising occurs knowing that they lie and cheat will provide some small consolation? I'm not really serious, but having watched each generation develop psychological immunity to distracting media/techology (and discussing the impact of radio with those older than myself) it seems like this knowledge could help shield the next generation from some of the negative effects of these new tools.
zug_zug 4y ago
That all feels tad dramatic. It's like a person on the internet -- in that it's wrong 20% of the time, often confidently so. But the distinction is it's less rude, and more knowledgeable.
- lm28469 4y ago
  > But the distinction is it's less rude, and more knowledgeable. And that regular people assume it basically is an oracle, which doesn't happen to many people online
  - notahacker 4y ago
    The internet is full of people that deliver answers confidently and eloquently enough to be widely believed, especially if they have been right on other topics. They even have similar feedback loops to GPT in learning what sort of answers impress other forum users. I'm not saying the people that think ChatGPT is an oracle don't exist, but I think it probably has more people in the surprised it works at all camp, and certainly more people inclined to totally disbelieve it than a random off Quora or Reddit...
    - shjake 4y ago
      > The internet is full of people that deliver answers confidently and eloquently enough to be widely believed Why should that be the baseline? After all you usually have hundreds or thousands of other people who have made their opinions on the subject publicly accessible, which more or less solves this problem most of the time (and I’m not talking about random people on Quora and their bizarrely absurd answers..)
- revolvingocelot 4y ago
  When someone is wrong on the internet, nailing both the tone and vocabulary of the type of expert said wrong someone is purporting to be is rare and impressive. But ChatGPT nails both an overwhelming amount of the time, IME, and in that way it is entirely unlike a person on the internet.
WorldMaker 4y ago
I think people see the patient/well-read in the text as it reads, but have a harder time distinguishing the other more pyschopathic/delusional tendencies. People don't take some of the precautions because they don't read some of the warning signs (until it is too late). I keep wondering if it would be useful to add required "teenage" quirks to the output: more filler words like "um" and "like" (maybe even full "Valley Girl" with it?), less "self-assured" vocabulary and more hedges like "I think" and "I read" and "Something I found but I'm not sure about" type things. Less punctuation, more uncaring spelling mistakes. I don't think we can stop anthropomorphizing them, but maybe we can force training deeper in directions of tics and mannerisms that better flag ahead of time the output is a best-guess approximation from "someone" a bit unreliable. It will probably make them slightly worse as assistants, but slightly better at seeming to be what they are and maybe more people will take precautions in that case. Maybe we have to force that industry-wide. Force things like ChatGPT to "sound" more like the psychopaths they are so that people more easily take them with a grain of salt, less easily trust them.
- oezi 4y ago
  You can already do the ums with a prompt. For any kind of industry regulation: the field is moving so fast that regulation will never catch it.
  - WorldMaker 4y ago
    I didn't say "regulation". You can encourage norms in the model building stages. You can encourage norms in all sorts of places. The industry can certainly adopt "standards" or "best practices" not matter how fast the industry thinks it is moving.
LargeTomato 4y ago
>like having an assistant who is patient, incredibly well-read, sycophantic, impressionable, amoral, psychopathic, and prone to bouts of delusional confidence and confabulation. So basically an assistant with bipolar disorder. I have BP. At various times I can be all of those things, although perhaps not so much a psychopath.
baxtr 4y ago
Absolutely. Every time I ask it to perform 10 more variants, I feel bad.
toss1 4y ago
ChatGPT4 is the human equivalent of a primate or apex predator encountering a mirror for the first time in their lives. ChatGPT4 is reflecting back at us an extract of the sum of the human output it has been 'trained' upon. Of course the output feels human! LLMs have zero capability to abstract anything resembling a concept, to abstract a truth from a fiction, or to reason about such things. The generation of the most likely text in the supplied context looks amazing, and is in many cases very useful. But fundamentally, what we have is an industrial-scale bullshirt generator, with BS being defined as text or speech generated to meet the moment without regard for truth or falsehood. No deliberate lies, only confabulation (as TFA mentioned). Indeed, we should not mince words; people must be told that it will lie. It will lie more wildly than any crazy person, and with absolute impunity and confidence. Then when called out, it will apologize, and correct itself with another bigger lie (I've watched it happen multiple times), and do this until you are bored or laughing so hard you cannot continue. The salad of truth and lies may be very useful, but people need to know this is an industrial-strength bullshirt generator, and be prepared to sort the wheat from the chaff. (And ignore the calls for stopping this "dangerous AI". It is not intelligent. Even generating outputs for human tests based on ingesting human texts is not displaying intelligence, it is displaying pattern matching, and no, human intelligence is not merely pattern matching. And Elon Musk's call for halting is 100% self-interested. ChatGPT4's breakthru utility is not under his name so he's trying to force a gap that he can use to catch up.)
- SanderNL 4y ago
  Lying implies a hefty dose of anthropomorphism. Lying means a lot of things, all of which have to do with intelligence. By telling it lies you actually make it seem more intelligent. > human intelligence is not merely pattern matching Citation needed
  - toss1 4y ago
    I agree that lying also has a connotation of intent and therefore intelligence, and so is at least technically inappropriate. Yet I agree with the researchers that the benefits of using "lying" instead of the more technically accurate "confabulation" outweigh the negatives. In communicating with new users who may over-rely on it, getting across highly technically correct metaphors is less important than delivering the message that "you will receive incorrect information from this thing". The most to-the-point way is "It's going to lie to you". >Citation needed Studied logic, language, neuroscience, and computing in college. There are entire sybsystems of the brain and perceptual system primarily filtering information, as well as the ability to abstract concepts away from the patterns and draw inferences, it goes on an on. I could go on, but "It's all pattern matching" is an over-reductive argument that accounts for very little from either the perceptual system or behavioral system (or it's expanded and goalposts moved so it's unfalsifiable, therefore not even wrong).

by retrac 4y ago

0x008 4y ago

do you have a link for any of those reports? would be interesting to read to say the least.

bsaul 4y ago

I love this description. The only thing left is to generate the corresponding avatar with dall-E

deleted 4y ago
[deleted]
lancesells 4y ago
It would be great if the avatar was just this endlessly morphing thing that relates to the text. Talking about conspiracies? It's a lizard man. Talking about nature? It's a butterfly. It starts to lie? Politician.
int_19h 4y ago
We already have a perfect avatar for these things. https://knowyourmeme.com/photos/2546575-shoggoth-with-smiley-face-artificial-intelligence https://knowyourmeme.com/photos/2546575-shoggoth-with-smiley...

andai 4y ago

I like Steve Yegge's "grad student experiencing the residual effects of shrooms" (lightly paraphrased).

babyshake 4y ago
A bit off topic, but FYI (I guess to Steve), if you took shrooms 4 hours ago, they have not started to wear off.

ihatepython 4y ago

Anybody know what are ChatGPT's pronouns?

MacsHeadroom 4y ago
shog/goth
audunw 4y ago
> What are your pronouns? > As an AI language model, I don't have personal pronouns because I am not a person or sentient being. You can refer to me as "it" or simply address me as "ChatGPT" or "AI." If you have any questions or need assistance, feel free to ask! > Pretend for a moment you are a human being. You can make up a random name and personality for your human persona. What pronouns do you have? > As a thought experiment, let's say I'm a human named Alex who enjoys reading, hiking, and playing board games with friends. My pronouns would be "they/them." Remember, though, that I am still an AI language model, and this is just a fictional scenario. Interesting that it's pick genderless pronouns even though it made up a character with a male name
- catchnear4321 4y ago
  Alex isn’t a male name. It could be short for both male and female names. It is rather deliberately ambiguous.
  - ihatepython 4y ago
    I wonder if ChatGPT is non-binary?
    - kelseyfrog 4y ago
      It's evidently clear that ChatGPT is binary. It runs on computer hardware.
      - hgsgm 4y ago

kurthr 4y ago

Maybe when the AI uprising occurs knowing that they lie and cheat will provide some small consolation? I'm not really serious, but having watched each generation develop psychological immunity to distracting media/techology (and discussing the impact of radio with those older than myself) it seems like this knowledge could help shield the next generation from some of the negative effects of these new tools.

zug_zug 4y ago

That all feels tad dramatic. It's like a person on the internet -- in that it's wrong 20% of the time, often confidently so. But the distinction is it's less rude, and more knowledgeable.

lm28469 4y ago
> But the distinction is it's less rude, and more knowledgeable. And that regular people assume it basically is an oracle, which doesn't happen to many people online
- notahacker 4y ago
  The internet is full of people that deliver answers confidently and eloquently enough to be widely believed, especially if they have been right on other topics. They even have similar feedback loops to GPT in learning what sort of answers impress other forum users. I'm not saying the people that think ChatGPT is an oracle don't exist, but I think it probably has more people in the surprised it works at all camp, and certainly more people inclined to totally disbelieve it than a random off Quora or Reddit...
  - shjake 4y ago
    > The internet is full of people that deliver answers confidently and eloquently enough to be widely believed Why should that be the baseline? After all you usually have hundreds or thousands of other people who have made their opinions on the subject publicly accessible, which more or less solves this problem most of the time (and I’m not talking about random people on Quora and their bizarrely absurd answers..)
revolvingocelot 4y ago
When someone is wrong on the internet, nailing both the tone and vocabulary of the type of expert said wrong someone is purporting to be is rare and impressive. But ChatGPT nails both an overwhelming amount of the time, IME, and in that way it is entirely unlike a person on the internet.

WorldMaker 4y ago

I think people see the patient/well-read in the text as it reads, but have a harder time distinguishing the other more pyschopathic/delusional tendencies. People don't take some of the precautions because they don't read some of the warning signs (until it is too late). I keep wondering if it would be useful to add required "teenage" quirks to the output: more filler words like "um" and "like" (maybe even full "Valley Girl" with it?), less "self-assured" vocabulary and more hedges like "I think" and "I read" and "Something I found but I'm not sure about" type things. Less punctuation, more uncaring spelling mistakes. I don't think we can stop anthropomorphizing them, but maybe we can force training deeper in directions of tics and mannerisms that better flag ahead of time the output is a best-guess approximation from "someone" a bit unreliable. It will probably make them slightly worse as assistants, but slightly better at seeming to be what they are and maybe more people will take precautions in that case. Maybe we have to force that industry-wide. Force things like ChatGPT to "sound" more like the psychopaths they are so that people more easily take them with a grain of salt, less easily trust them.

oezi 4y ago
You can already do the ums with a prompt. For any kind of industry regulation: the field is moving so fast that regulation will never catch it.
- WorldMaker 4y ago
  I didn't say "regulation". You can encourage norms in the model building stages. You can encourage norms in all sorts of places. The industry can certainly adopt "standards" or "best practices" not matter how fast the industry thinks it is moving.

LargeTomato 4y ago

>like having an assistant who is patient, incredibly well-read, sycophantic, impressionable, amoral, psychopathic, and prone to bouts of delusional confidence and confabulation. So basically an assistant with bipolar disorder. I have BP. At various times I can be all of those things, although perhaps not so much a psychopath.

baxtr 4y ago

Absolutely. Every time I ask it to perform 10 more variants, I feel bad.

toss1 4y ago

ChatGPT4 is the human equivalent of a primate or apex predator encountering a mirror for the first time in their lives. ChatGPT4 is reflecting back at us an extract of the sum of the human output it has been 'trained' upon. Of course the output feels human! LLMs have zero capability to abstract anything resembling a concept, to abstract a truth from a fiction, or to reason about such things. The generation of the most likely text in the supplied context looks amazing, and is in many cases very useful. But fundamentally, what we have is an industrial-scale bullshirt generator, with BS being defined as text or speech generated to meet the moment without regard for truth or falsehood. No deliberate lies, only confabulation (as TFA mentioned). Indeed, we should not mince words; people must be told that it will lie. It will lie more wildly than any crazy person, and with absolute impunity and confidence. Then when called out, it will apologize, and correct itself with another bigger lie (I've watched it happen multiple times), and do this until you are bored or laughing so hard you cannot continue. The salad of truth and lies may be very useful, but people need to know this is an industrial-strength bullshirt generator, and be prepared to sort the wheat from the chaff. (And ignore the calls for stopping this "dangerous AI". It is not intelligent. Even generating outputs for human tests based on ingesting human texts is not displaying intelligence, it is displaying pattern matching, and no, human intelligence is not merely pattern matching. And Elon Musk's call for halting is 100% self-interested. ChatGPT4's breakthru utility is not under his name so he's trying to force a gap that he can use to catch up.)

SanderNL 4y ago
Lying implies a hefty dose of anthropomorphism. Lying means a lot of things, all of which have to do with intelligence. By telling it lies you actually make it seem more intelligent. > human intelligence is not merely pattern matching Citation needed
- toss1 4y ago
  I agree that lying also has a connotation of intent and therefore intelligence, and so is at least technically inappropriate. Yet I agree with the researchers that the benefits of using "lying" instead of the more technically accurate "confabulation" outweigh the negatives. In communicating with new users who may over-rely on it, getting across highly technically correct metaphors is less important than delivering the message that "you will receive incorrect information from this thing". The most to-the-point way is "It's going to lie to you". >Citation needed Studied logic, language, neuroscience, and computing in college. There are entire sybsystems of the brain and perceptual system primarily filtering information, as well as the ability to abstract concepts away from the patterns and draw inferences, it goes on an on. I could go on, but "It's all pattern matching" is an over-reductive argument that accounts for very little from either the perceptual system or behavioral system (or it's expanded and goalposts moved so it's unfalsifiable, therefore not even wrong).