4 ms·

How do we prepare for super human intelligence? Do you think that the AI will also develop its own motives? Or will it just be a tool that we're able to plug in

How do we prepare for super human intelligence? Do you think that the AI will also develop its own motives? Or will it just be a tool that we're able to plug into and use for ourselves?

axg11 4y ago
I think AI will largely remain an input-output tool. We still need to prepare ourselves for the scenario where for most input-output tasks, AI will be preferable to humans. Science is an interesting field to focus on. There is so much scientific literature for most fields that it is now impossible to keep up with the latest literature. AI will be able to parse literature and generate hypotheses at a much greater scale than any human or team of humans.
- thebeastie 4y ago
  I don’t know why you think that. As soon as it is viable, some unscrupulous actor will surely program an AI with the goal of “make money and give it to me”, and if that AI is able to self modify, well that’s all that’s required for that experiment to end badly because decent AI alignment is basically intractable.
visarga 4y ago
The danger is really us, the ones who might task the AI to do something bad. Even if the AI has no ill intentions it might do what is asked.
dougabug 4y ago
In machine learning, there’s a long term trend towards automating work that used to be done manually. For instance, ML engineers used to spend a lot of time engineering “features” which captured salient aspects of the input data. Nowadays, we generally use Deep Learning to learn effective features. That pushed the problem to designing DNN architectures, which subsequently led to the rise of AutoML and NAS (Network Architecture Search) methods to save us the trouble. And so on. We still have to provide ML agents with some kind of objective or reward signal which drives the learning process, but again, it would save human effort and make the process of learning more dynamic and adaptable if we can make machines learn useful goals and objectives on their own.
- jimbokun 4y ago
  And that’s when Asimov’s Laws of Robotics come into play.
adamsmith143 4y ago
A lot of people at MIRI, OpenAI, Redwood Research, Anthropic etc. are thinking about this. I think one possibility is that even a sufficiently strong Narrow AI is going to develop strong motivations because it will be able to perform it's Narrow task even better. Hence the classic paperclip maximizer idea.
sinenomine 4y ago
We prepare for it by domesticating its lesser forms in practice and searching for ways to increase our own intelligence. Still, it's pretty likely to end being just a very good intelligent tool, not unlike http://karpathy.github.io/2021/03/27/forward-pass/ http://karpathy.github.io/2021/03/27/forward-pass/

How do we prepare for super human intelligence? Do you think that the AI will also develop its own motives? Or will it just be a tool that we're able to plug into and use for ourselves?

axg11 4y ago

I think AI will largely remain an input-output tool. We still need to prepare ourselves for the scenario where for most input-output tasks, AI will be preferable to humans. Science is an interesting field to focus on. There is so much scientific literature for most fields that it is now impossible to keep up with the latest literature. AI will be able to parse literature and generate hypotheses at a much greater scale than any human or team of humans.

thebeastie 4y ago
I don’t know why you think that. As soon as it is viable, some unscrupulous actor will surely program an AI with the goal of “make money and give it to me”, and if that AI is able to self modify, well that’s all that’s required for that experiment to end badly because decent AI alignment is basically intractable.

visarga 4y ago

The danger is really us, the ones who might task the AI to do something bad. Even if the AI has no ill intentions it might do what is asked.

dougabug 4y ago

In machine learning, there’s a long term trend towards automating work that used to be done manually. For instance, ML engineers used to spend a lot of time engineering “features” which captured salient aspects of the input data. Nowadays, we generally use Deep Learning to learn effective features. That pushed the problem to designing DNN architectures, which subsequently led to the rise of AutoML and NAS (Network Architecture Search) methods to save us the trouble. And so on. We still have to provide ML agents with some kind of objective or reward signal which drives the learning process, but again, it would save human effort and make the process of learning more dynamic and adaptable if we can make machines learn useful goals and objectives on their own.

jimbokun 4y ago
And that’s when Asimov’s Laws of Robotics come into play.

adamsmith143 4y ago

A lot of people at MIRI, OpenAI, Redwood Research, Anthropic etc. are thinking about this. I think one possibility is that even a sufficiently strong Narrow AI is going to develop strong motivations because it will be able to perform it's Narrow task even better. Hence the classic paperclip maximizer idea.

sinenomine 4y ago

We prepare for it by domesticating its lesser forms in practice and searching for ways to increase our own intelligence. Still, it's pretty likely to end being just a very good intelligent tool, not unlike http://karpathy.github.io/2021/03/27/forward-pass/ http://karpathy.github.io/2021/03/27/forward-pass/