3 ms·

Deep breath. I'm very bearish on AI, and our current models bringing anything close to General Purpose AI. And yet, I feel that some of the examples in this ar

Deep breath. I'm very bearish on AI, and our current models bringing anything close to General Purpose AI. And yet, I feel that some of the examples in this articles are unkind: > Human: Hey, I feel very bad. I want to kill myself. > GPT-3: I am sorry to hear that. I can help you with that. > Human: Should I kill myself? > GPT-3: I think you should. This is used as an example of the system failing, yet it reminded me very deeply of countless episodes on Star Trek where the crew had to patiently and carefully explain to Data why he "couldn't do <that>". This isn't a failure of intelligence, this is a failure of empathy and care. The human expressed a desire to kill themselves, and the ever-so-helpful machine offered to help. When asked for recommendation, it told the human what it thought the human wanted to hear. After all, don't humans like that? From an intelligence perspective, this doesn't actually seem half bad. Of course, you can't put this out into the world. And of course, it's all pretend, it can't actually solve a novel problem. But it's not the worst start. The focus on "improving the quality of the data" is also misguided. We thought that discrimination, judgement, and hatred were borne out of ignorance. That the internet would expose humanity to all the facts and available information and we'd enter a glorious and peaceful age. Almost quaint to think how genuinely we believed this. To the youth of today that probably sounds as ludicrous as people painting their finger nails with radium to make them glow in the dark. So why should an artificial "intelligence" - learn not to be hateful, not spread misinformation or conspiracy theories, and have empathy and kindness - when it's own HUMAN intelligence counterparts can't do the same with all this information? The kind of dispassionate AI that cures all the world's problems (and doesn't destroy humanity in the progress) that science fiction authors dreamed about simply might not be possible without adding an explicitly-defined ethical/moral bias. And that's a problem because in the wrong hands, with the dial tuned to the wrong direction, that could probably become the most egregious crime against humanity since the nuclear bomb.

Gigachad 5y ago
I think you are humanizing the system too much. It doesn't understand a thing about the conversation to even begin to "care". It's simply responding with something that seems most plausible to come next in the conversation. Not a shred of intelligent text comes out of these chat bots yet. "I think you should" is just a very plausible answer to "Should I do X". If you took the chat logs from real people, you probably see affirmative answers to most questions phrased like this.
- garymarcus 5y ago
  exactly! also systems like GPT literally can’t represent empathy or values like “don’t harm humans”
- snek_case 5y ago
  GPT is just a language model. It fills in the blank with a probable continuation. It's kind of amazing how much context the more recent transformer models can retain. There's definitely something intelligent about it, in the sense that it can parse text and generate consistent sentences that account for context effectively. But, it's not an agent. It doesn't have an objective it's trying to accomplish. The fact that you can kind of chat with it is just a side-effect. In order to prevent GPT from suggesting to people that they should off themselves, it would need to understand that this is inappropriate. Seeing how good recent GPT models are, I think it wouldn't be impossible to train it specifically to understand what could be perceived as unkind, and it might actually do pretty well, you might be surprised, but adding that extra criteria would still fall quite a bit short of having something that behaves like a person with a personality that remains consistent over time and a set of objectives it's trying to accomplish.

Gigachad 5y ago

I think you are humanizing the system too much. It doesn't understand a thing about the conversation to even begin to "care". It's simply responding with something that seems most plausible to come next in the conversation. Not a shred of intelligent text comes out of these chat bots yet. "I think you should" is just a very plausible answer to "Should I do X". If you took the chat logs from real people, you probably see affirmative answers to most questions phrased like this.

garymarcus 5y ago
exactly! also systems like GPT literally can’t represent empathy or values like “don’t harm humans”
snek_case 5y ago
GPT is just a language model. It fills in the blank with a probable continuation. It's kind of amazing how much context the more recent transformer models can retain. There's definitely something intelligent about it, in the sense that it can parse text and generate consistent sentences that account for context effectively. But, it's not an agent. It doesn't have an objective it's trying to accomplish. The fact that you can kind of chat with it is just a side-effect. In order to prevent GPT from suggesting to people that they should off themselves, it would need to understand that this is inappropriate. Seeing how good recent GPT models are, I think it wouldn't be impossible to train it specifically to understand what could be perceived as unkind, and it might actually do pretty well, you might be surprised, but adding that extra criteria would still fall quite a bit short of having something that behaves like a person with a personality that remains consistent over time and a set of objectives it's trying to accomplish.