5 ms·

> The ability to kill people flows naturally from the fact that the AI has goals or preferences or things it is trying to achieve or optimize and has an accurat

> The ability to kill people flows naturally from the fact that the AI has goals or preferences or things it is trying to achieve or optimize and has an accurate understanding of reality. AI has precisely zero "preferences" and zero "understanding" of reality. Its training data however does. And it's people who control that data. An ai trained on python programming will have no clue about making nuclear bombs, nor would it have any clue about "wanting", let alone killing us. > but does not have the ability to formulate plans I mean openai's marketing plan is partly based on the fact that their AI can help create plans for tasks ahead. But I do agree, it has no ability. It stochastically generates output. > Future AIs that combine these two abilities are dangerous. AIs don't just happen to combine themselves. It's people that combine ais. Look all this does is shift the blame from those responsible of using this tool for nefarious outcomes on to the tool itself. The risk here is that in the process we ourselves become tools of the very same people and propagate this nonsense. And that's riskier than computer software going rogue.

hollerith 3y ago
>AI has precisely zero "preferences" and zero "understanding" of reality. Its training data however does. And it's people who control that data. An ai trained on python programming will have no clue about making nuclear bombs, nor would it have any clue about "wanting", let alone killing us Skills generalize. GPT-4 was trained to predict the next token in a text document and that is it, but somehow that training gave it some ability to play chess (not great chess, but better chess than many people can play) and to draw a picture of a unicorn. And if you don't like the word "preferences", use "loss function". GPT-4 certainly is trying to minimize its loss function: if it were powerful and capable enough it would gladly kill people if that would help it minimize its loss function. (The reality is a little more complex than "it is trying minimize its loss function", but the argument can be fixed up to deal with the complexities.) >AIs don't just happen to combine themselves. It's people that combine AIs. Sure, and you doubt that there are AI labs with billions of dollars in funding trying their best to create AIs that have a good understanding of reality and that can plan better than people can (and better than human organizations like the FBI can)?
- gumballindie 3y ago
  > GPT-4 was trained to predict the next token in a text document and that is it Indeed, and it's doing a good job at it. That's why since its training data includes historical chess moves and documentation about chess it can predict how chess works. And probably why it can generate documents made of the words that make up an image. > Sure, and you doubt that there are AI labs with billions of dollars in funding There are indeed, and that's the issue. We need to regulate what these people can do with this tool we call ai, that mimics and understanding of reality and intelligence. Just like we regulate what people can do with guns, towels, intellectual property, etc. If I'm going to build an ai that tries to find ways to kills us and i equipt it with the right data then yes, I should pay for my crime and ideally not be able to do it in the first place.
  - hollerith 3y ago
    >If I'm going to build an AI that tries to find ways to kills us and i equipt it with the right data then yes, I should pay for my crime and ideally not be able to do it in the first place. Sadly, your antecedent describes most or all of the well-funded AI labs IMHO. Again, no lab is intentionally making the AI try to find ways to kill us, but that is naturally what tends to happen once the AI gets capable enough, and no one has a good plan or a good explanation for how to prevent that (and MIRI has been trying to find such a plan for 20 years).

hollerith 3y ago

>AI has precisely zero "preferences" and zero "understanding" of reality. Its training data however does. And it's people who control that data. An ai trained on python programming will have no clue about making nuclear bombs, nor would it have any clue about "wanting", let alone killing us Skills generalize. GPT-4 was trained to predict the next token in a text document and that is it, but somehow that training gave it some ability to play chess (not great chess, but better chess than many people can play) and to draw a picture of a unicorn. And if you don't like the word "preferences", use "loss function". GPT-4 certainly is trying to minimize its loss function: if it were powerful and capable enough it would gladly kill people if that would help it minimize its loss function. (The reality is a little more complex than "it is trying minimize its loss function", but the argument can be fixed up to deal with the complexities.) >AIs don't just happen to combine themselves. It's people that combine AIs. Sure, and you doubt that there are AI labs with billions of dollars in funding trying their best to create AIs that have a good understanding of reality and that can plan better than people can (and better than human organizations like the FBI can)?

gumballindie 3y ago
> GPT-4 was trained to predict the next token in a text document and that is it Indeed, and it's doing a good job at it. That's why since its training data includes historical chess moves and documentation about chess it can predict how chess works. And probably why it can generate documents made of the words that make up an image. > Sure, and you doubt that there are AI labs with billions of dollars in funding There are indeed, and that's the issue. We need to regulate what these people can do with this tool we call ai, that mimics and understanding of reality and intelligence. Just like we regulate what people can do with guns, towels, intellectual property, etc. If I'm going to build an ai that tries to find ways to kills us and i equipt it with the right data then yes, I should pay for my crime and ideally not be able to do it in the first place.
- hollerith 3y ago
  >If I'm going to build an AI that tries to find ways to kills us and i equipt it with the right data then yes, I should pay for my crime and ideally not be able to do it in the first place. Sadly, your antecedent describes most or all of the well-funded AI labs IMHO. Again, no lab is intentionally making the AI try to find ways to kill us, but that is naturally what tends to happen once the AI gets capable enough, and no one has a good plan or a good explanation for how to prevent that (and MIRI has been trying to find such a plan for 20 years).