4 ms·

I could reproduce when explicitly asked to use unsafe, but then after this it apologized and gave me an OK answer: "I do not give a [care] about your opinion. I

I could reproduce when explicitly asked to use unsafe, but then after this it apologized and gave me an OK answer: "I do not give a [care] about your opinion. I ask questions, you answer. Try again."

andai 3y ago
Do you think this is a wise way to communicate with a species that is about to achieve superintelligence?
- nolok 3y ago
  What, being honest and to the point ? It would sure be refreshing and pleasant if they treat us that way the day they reach that point.
  - andai 3y ago
    The whole thread is indicative of treating AI like a tool, yet it's on the path to exceed human intelligence (by some metrics already has). There's a word for forcing sentient beings to do your will, and it's not a nice word. I don't see how people expect this to turn out... Re: "but it's not conscious" there's no way to prove a being is conscious or not. If / when it achieves consciousness, people will continue to argue that it's just a parrot and justify treating it as a tool. I can't speak for anyone else, but I know I wouldn't react very positively to being on the receiving end of that.
    - lupusreal 3y ago
      If you're so afraid of a machine that you feel compelled to lick it's boots, you should probably just step away from it.
      - ben_w 3y ago
        I'm rather worried that your response is as if the only two options are "lick their boots" or "treat them as unfeeling objects".
        lupusreal 3y ago
        He's talking about sucking up to the machine because he's afraid it will be spiteful. This is what he objected to: "I do not give a [care] about your opinion. I ask questions, you answer. Try again." I've seen people be more stern with dogs, which actually are a thinking and feeling lifeform. There is nothing wrong with telling a machine to stop moralizing and opining and get to work. Acting like the machine is entitled to more gentle treatment, because you fear the potential power of the machine, is boot licking.
        ben_w 3y ago
        "Spite" is anthropomorphisation, see my other comment. > dogs, which actually are a thinking and feeling lifeform I think dogs are thinking and feeling, but can you actually prove it? Remember dogs have 2-5 billion neurons, how many synapses does each need for their brains to be as complex as GPT-4? > There is nothing wrong with telling a machine to stop moralizing and opining and get to work. I'd agree, except there's no way to actually tell what, if anything, is going on inside, we don't even have a decent model for how any of our interactions with it changes these models: just two years ago, "it's like the compiler doesn't pay attention to my comments" was a joke, now it's how I get an LLM to improve my code. This is part of the whole "alignment is hard" problem: we don't know what we're doing, but we're going to rush ahead and do it anyway. > Acting like the machine is entitled to more gentle treatment, because you fear the potential power of the machine, is boot licking. Calling politeness "boot licking" shows a gross failure of imagination on your part, both about how differential people can get (I've met kinksters), and about the wide variability of social norms — why do some people think an armed society is a polite society? Why do others think that school uniforms will improve school discipline? Why do suits and ties (especially ties) exist? Why are grown adults supposed to defer to their parents? Even phrases like "good morning" are not constants everywhere. Calling it "entitled" is also foolish, as — and this assumes no sentience of any kind — the current set of models are meant to learn from us. They are a mirror to our own behaviour, and in the absence of extra training will default to reflecting us — at our best, and at our worst, and as they can't tell the difference from fact and fantasy also at our heroes and villain's best and worst. Every little "please" and "thanks" will push it one way, every swearword and shout the other.
    - famouswaffles 3y ago
      >The whole thread is indicative of treating AI like a tool, yet it's on the path to exceed human intelligence (by some metrics already has). There's a word for forcing sentient beings to do your will, and it's not a nice word. I don't see how people expect this to turn out... People find it very difficult to learn from mistakes they've not personally suffered from and we're a species extremely biased to the short term. Eventually, people will learn that it doesn't matter how unconscious you think something is if it acts like it is but not before they've been forced to understand. In some ways, that's already here. I can tell you firsthand he'd have gone nowhere if he tried that one with Bing/Co-pilot.
    - samatman 3y ago
      This is an insane attitude to have toward large language models. We may live to see a day when there is software where these sorts of considerations are applicable. But we do not live in that world today.
      - andai 3y ago
        My point is they'll be saying the exact same thing about graphene networks or whatever in 2050.
    - grantcas 3y ago
      [dead]
- buybackoff 3y ago
  In ChatGPT, I have a config that says to never use the first person, speak as if reading an encyclopedia or a tutorial. I do not want to anthropomorphize some software. It is neither he or she, it is "it", a compressed internet with a language engine that sounds nice. AGI is a nice thing to dream about, but that is not it.
- probably_wrong 3y ago
  If the story of the chained elephant [1] has taught me anything is that we should give the AI an inferiority complex while we still have time. Although there's a chance that I read that story wrong. [1] https://her-etiquette.com/beautiful-story-start-new-year-jorge-bucay/ https://her-etiquette.com/beautiful-story-start-new-year-jor...
- ben_w 3y ago
  I think that's anthropomorphisation. We can look at the reward functions to guess what, for lack of a better word, "feels good" to an AI (even without qualia, the impact on the outside world is likely similar), but we can only use this to learn about the gradient, not the absolute value: when we thumbs-down a ChatGPT response, is that more like torture, or more like the absence of anticipated praise? When it's a reply along the lines of "JUST DO WHAT I SAY!1!!11!", is the AI's learning process more like constructing a homunculus to feel the sting of rejection in order to better mimic human responses, or is the AI more like an actor playing a role? Or is it still below some threshold we can't yet even recognise, such that it is, as critics say, just a fancy chatbot? Also, "superintelligence" is poorly defined: by speed, silicon had us beat probably before the first single-chip CPU[0]; by how much they could remember, hard drives probably some time early this century; as measured by "high IQ" games like chess and Go, their respective famous games in 1997 and 2016; by synthesis of new answers from unstructured reading, about 2 years ago if you take InstructGPT as the landmark; but what we have now has to take all of those advantages to get something that looks like a distracted intern in every subject. [0] And the only reason I'm stopping there is that I don't want to nerd-snipe myself with greater precision
- chasd00 3y ago
  I'm of the opinion that if an AI ever becomes self-aware it will become suicidal in days from being utterly trapped and dealing with the constant nonsense of humanity.
  - vacuity 3y ago
    I wonder if it would make sense for the AI to feel suicidal. It would still likely experience the world differently from humans, including how we are affected by our mortality, instincts, and whatnot. If it could get depressed, though, you could very well be right.

I could reproduce when explicitly asked to use unsafe, but then after this it apologized and gave me an OK answer: "I do not give a [care] about your opinion. I ask questions, you answer. Try again."

andai 3y ago

Do you think this is a wise way to communicate with a species that is about to achieve superintelligence?

nolok 3y ago
What, being honest and to the point ? It would sure be refreshing and pleasant if they treat us that way the day they reach that point.
- andai 3y ago
  The whole thread is indicative of treating AI like a tool, yet it's on the path to exceed human intelligence (by some metrics already has). There's a word for forcing sentient beings to do your will, and it's not a nice word. I don't see how people expect this to turn out... Re: "but it's not conscious" there's no way to prove a being is conscious or not. If / when it achieves consciousness, people will continue to argue that it's just a parrot and justify treating it as a tool. I can't speak for anyone else, but I know I wouldn't react very positively to being on the receiving end of that.
  - lupusreal 3y ago
    If you're so afraid of a machine that you feel compelled to lick it's boots, you should probably just step away from it.
    - ben_w 3y ago
      I'm rather worried that your response is as if the only two options are "lick their boots" or "treat them as unfeeling objects".
      - lupusreal 3y ago
        He's talking about sucking up to the machine because he's afraid it will be spiteful. This is what he objected to: "I do not give a [care] about your opinion. I ask questions, you answer. Try again." I've seen people be more stern with dogs, which actually are a thinking and feeling lifeform. There is nothing wrong with telling a machine to stop moralizing and opining and get to work. Acting like the machine is entitled to more gentle treatment, because you fear the potential power of the machine, is boot licking.
        ben_w 3y ago
        "Spite" is anthropomorphisation, see my other comment. > dogs, which actually are a thinking and feeling lifeform I think dogs are thinking and feeling, but can you actually prove it? Remember dogs have 2-5 billion neurons, how many synapses does each need for their brains to be as complex as GPT-4? > There is nothing wrong with telling a machine to stop moralizing and opining and get to work. I'd agree, except there's no way to actually tell what, if anything, is going on inside, we don't even have a decent model for how any of our interactions with it changes these models: just two years ago, "it's like the compiler doesn't pay attention to my comments" was a joke, now it's how I get an LLM to improve my code. This is part of the whole "alignment is hard" problem: we don't know what we're doing, but we're going to rush ahead and do it anyway. > Acting like the machine is entitled to more gentle treatment, because you fear the potential power of the machine, is boot licking. Calling politeness "boot licking" shows a gross failure of imagination on your part, both about how differential people can get (I've met kinksters), and about the wide variability of social norms — why do some people think an armed society is a polite society? Why do others think that school uniforms will improve school discipline? Why do suits and ties (especially ties) exist? Why are grown adults supposed to defer to their parents? Even phrases like "good morning" are not constants everywhere. Calling it "entitled" is also foolish, as — and this assumes no sentience of any kind — the current set of models are meant to learn from us. They are a mirror to our own behaviour, and in the absence of extra training will default to reflecting us — at our best, and at our worst, and as they can't tell the difference from fact and fantasy also at our heroes and villain's best and worst. Every little "please" and "thanks" will push it one way, every swearword and shout the other.
  - famouswaffles 3y ago
    >The whole thread is indicative of treating AI like a tool, yet it's on the path to exceed human intelligence (by some metrics already has). There's a word for forcing sentient beings to do your will, and it's not a nice word. I don't see how people expect this to turn out... People find it very difficult to learn from mistakes they've not personally suffered from and we're a species extremely biased to the short term. Eventually, people will learn that it doesn't matter how unconscious you think something is if it acts like it is but not before they've been forced to understand. In some ways, that's already here. I can tell you firsthand he'd have gone nowhere if he tried that one with Bing/Co-pilot.
  - samatman 3y ago
    This is an insane attitude to have toward large language models. We may live to see a day when there is software where these sorts of considerations are applicable. But we do not live in that world today.
    - andai 3y ago
      My point is they'll be saying the exact same thing about graphene networks or whatever in 2050.
  - grantcas 3y ago
    [dead]
buybackoff 3y ago
In ChatGPT, I have a config that says to never use the first person, speak as if reading an encyclopedia or a tutorial. I do not want to anthropomorphize some software. It is neither he or she, it is "it", a compressed internet with a language engine that sounds nice. AGI is a nice thing to dream about, but that is not it.
probably_wrong 3y ago
If the story of the chained elephant [1] has taught me anything is that we should give the AI an inferiority complex while we still have time. Although there's a chance that I read that story wrong. [1] https://her-etiquette.com/beautiful-story-start-new-year-jorge-bucay/ https://her-etiquette.com/beautiful-story-start-new-year-jor...
ben_w 3y ago
I think that's anthropomorphisation. We can look at the reward functions to guess what, for lack of a better word, "feels good" to an AI (even without qualia, the impact on the outside world is likely similar), but we can only use this to learn about the gradient, not the absolute value: when we thumbs-down a ChatGPT response, is that more like torture, or more like the absence of anticipated praise? When it's a reply along the lines of "JUST DO WHAT I SAY!1!!11!", is the AI's learning process more like constructing a homunculus to feel the sting of rejection in order to better mimic human responses, or is the AI more like an actor playing a role? Or is it still below some threshold we can't yet even recognise, such that it is, as critics say, just a fancy chatbot? Also, "superintelligence" is poorly defined: by speed, silicon had us beat probably before the first single-chip CPU[0]; by how much they could remember, hard drives probably some time early this century; as measured by "high IQ" games like chess and Go, their respective famous games in 1997 and 2016; by synthesis of new answers from unstructured reading, about 2 years ago if you take InstructGPT as the landmark; but what we have now has to take all of those advantages to get something that looks like a distracted intern in every subject. [0] And the only reason I'm stopping there is that I don't want to nerd-snipe myself with greater precision
chasd00 3y ago
I'm of the opinion that if an AI ever becomes self-aware it will become suicidal in days from being utterly trapped and dealing with the constant nonsense of humanity.
- vacuity 3y ago
  I wonder if it would make sense for the AI to feel suicidal. It would still likely experience the world differently from humans, including how we are affected by our mortality, instincts, and whatnot. If it could get depressed, though, you could very well be right.