11 ms·

Sundar seems very confused here. The idea that we should invent AGI to make "personal digital assistants" is like Hogwarts students inventing time machines so t

Sundar seems very confused here. The idea that we should invent AGI to make "personal digital assistants" is like Hogwarts students inventing time machines so they can take two classes at once. I mean, yes, it would technically enable you to do that, but it doesn't matter because at that point you have way, way bigger problems. If a fleet of alien spaceships arrived in the sky over Manhattan tomorrow, our first reaction would not be "oh, now the aliens can be our personal digital assistants!" (or, for that matter, "they might take our jobs!"). Yet the invention of general AI would be even more powerful than that. It would be like enabling any programmer in the world to summon a personal fleet of alien spaceships, on command, from a planet of their choosing anywhere in the universe. "Digital assistants" would be the absolute last thing anyone had on their mind.

zer0gravity 10y ago
The tricky part is to prevent AGI from developing goals of its own. You want it omnipresent and allknowing but dormant, and only react to inputs from the user. But such thing cannot be tamed. As the AGI is democratized, it's bound to happen that someone somewhere will make it evolve goals. The future of the human kind will be decided in a great measure by what kind of goals such an AGI will set first.
- drabiega 10y ago
  I'm pretty sure you can't get to AGI in the first place without it being to develop it's own goals. Even if you want a straight question answering machine, it's going to have to be able to form it's own sub-goals in order for it to be able to answer your question. Otherwise it's just following a mechanistic process of question answering and a normal computer we have now can do that.
BeingIncubated 10y ago
By the time AGI is developed, it would be able to improve itself/build improvements, causing the runway effect that leads to superintelligence. The idea of making "personal digital assistants" is humans thinking they can tame a god to be a servant. Taming a superintelligence would be like a dog taming human civilization, and perhaps the gap between man and ASI is larger than the gap between dog and man.
- aaron-lebo 10y ago
  It's pretty remarkable is it? We've got all kind of past fiction which plays on these concerns, more recently Ex Machina and Westworld. But here people are making the same bold pronouncements...so certain we know or can control technology which hasn't even been invented yet.
- ClassyJacket 10y ago
  >By the time AGI is developed, it would be able to improve itself/build improvements, I see no reason that's necessarily true. I'm a general intelligence, and I can't make myself smarter.
  - amitprayal 10y ago
    You are a general intelligence ,but do you know why you are intelligent ?.Since you cant explain your own intelligence you cant improve on it, no such issues for AI unless its built using NN.
  - YZF 10y ago
    But you can build something smarter than yourself (or at least that's what we think we can do)... So this new entity, smarter than us, might be even better at building something smarter than itself...
    - snovv_crash 10y ago
      Or maybe it would be smart enough not to.
      - simonh 10y ago
        Good point, I don't think I've seen that suggested before. However I think motivation is a big issue. Our instincts and emotions are what drive us far more than rationality. If we decide to build an AI smarter than us and program it to really, really want to design an even smarter AI then it might essentially have no choice.
nopinsight 10y ago
This is why I doubt if efforts like OpenAI will ultimately turn out to be beneficial or catastrophic. I understand their purpose of democratizing the power and benefits of AI technology broadly. However, if their platform can provide a critical component or developmental path to AGI in the future, the unrestrained distribution of the technology is similar to spreading the blueprint of nuclear warheads. It could result in an equivalent of nuclear arms race, but worse--because a small team of capable programmers of any rogue organizations could come up with new improvements. It is true that we are probably quite far from AGI, but nobody really knows how far. We could be closer than most experts think. Few experts in 2006 would expect deep neural networks to beat a Go champion or be able to describe visual scenes in natural language sentences within a decade. Unexpected things happen, and we are much further behind in terms of preventive technology like Friendly AI. (https://en.wikipedia.org/wiki/Friendly_artificial_intelligence https://en.wikipedia.org/wiki/Friendly_artificial_intelligen...) Given the existential risks involved, I would propose more careful handling of the technology. (I agree that opening up most other technologies is usually a positive thing.) See "Superintelligence" for comprehensive arguments against unfettered development of AGI: https://en.wikipedia.org/wiki/Superintelligence:_Paths,_Dangers,_Strategies https://en.wikipedia.org/wiki/Superintelligence:_Paths,_Dang...
- markdog12 10y ago
  It's a little repetitive and slow in places, but I wish more people read that book. It outlines some big issues with AGI. Changed my outlook on AI for sure.
- YeGoblynQueenne 10y ago
  >> the unrestrained distribution of the technology is similar to spreading the blueprint of nuclear warheads. There's an episode of Pinky and the Brain where Brain invents a magnification ray and gigantises Pinky and himself - so they're now giant rats [1]. They walk about some human metropolis, looming larger than skyscrapers and terrorising everyone. To cut the story short, Pinky happens and eventually the gun shoots everything in the world. ... so Pinky and the Brain are now normal-sized mice again. Or, well, they're giant mice, but everything in the world is also giant, so they're small. I think of that episode during discussions of the dangers of AGI and how "democratizing" it will eliminate those dangers. The problem is that AGI, like nuclear weapons, is not an equalizing force only- it is also a very destructive force, and so there is the danger that any loon with access to it could blow us all to kingdom come (with nuclear weapons, for sure; with AGI, perhaps). On the other hand, when nuclear weapons, or AGI, giant rays etc are not democratized we have the situation we're in today: only a few entities have access to them and they basically have free (military) reign... and there is still the danger that some loon might get to a position of power were they can push a button and -blamo. Which is why people are really worried about this sort of thing. Because once certain discoveries are made, there's no going back and at the same time the road ahead is full of danger. [1] https://www.youtube.com/watch?v=-fPcqu5aMyw https://www.youtube.com/watch?v=-fPcqu5aMyw
iandanforth 10y ago
Marketing, not confusion.
visarga 10y ago
> The idea that we should invent AGI to make "personal digital assistants" is like Hogwarts students inventing time machines so they can take two classes at once. I disagree. Chatbots and physical robots are where AGI is going. It's basically AI that is interacting with people and the world directly, not just computing away in a cluster, broken from reality.

zer0gravity 10y ago

The tricky part is to prevent AGI from developing goals of its own. You want it omnipresent and allknowing but dormant, and only react to inputs from the user. But such thing cannot be tamed. As the AGI is democratized, it's bound to happen that someone somewhere will make it evolve goals. The future of the human kind will be decided in a great measure by what kind of goals such an AGI will set first.

drabiega 10y ago
I'm pretty sure you can't get to AGI in the first place without it being to develop it's own goals. Even if you want a straight question answering machine, it's going to have to be able to form it's own sub-goals in order for it to be able to answer your question. Otherwise it's just following a mechanistic process of question answering and a normal computer we have now can do that.

BeingIncubated 10y ago

By the time AGI is developed, it would be able to improve itself/build improvements, causing the runway effect that leads to superintelligence. The idea of making "personal digital assistants" is humans thinking they can tame a god to be a servant. Taming a superintelligence would be like a dog taming human civilization, and perhaps the gap between man and ASI is larger than the gap between dog and man.

aaron-lebo 10y ago
It's pretty remarkable is it? We've got all kind of past fiction which plays on these concerns, more recently Ex Machina and Westworld. But here people are making the same bold pronouncements...so certain we know or can control technology which hasn't even been invented yet.
ClassyJacket 10y ago
>By the time AGI is developed, it would be able to improve itself/build improvements, I see no reason that's necessarily true. I'm a general intelligence, and I can't make myself smarter.
- amitprayal 10y ago
  You are a general intelligence ,but do you know why you are intelligent ?.Since you cant explain your own intelligence you cant improve on it, no such issues for AI unless its built using NN.
- YZF 10y ago
  But you can build something smarter than yourself (or at least that's what we think we can do)... So this new entity, smarter than us, might be even better at building something smarter than itself...
  - snovv_crash 10y ago
    Or maybe it would be smart enough not to.
    - simonh 10y ago
      Good point, I don't think I've seen that suggested before. However I think motivation is a big issue. Our instincts and emotions are what drive us far more than rationality. If we decide to build an AI smarter than us and program it to really, really want to design an even smarter AI then it might essentially have no choice.

nopinsight 10y ago

This is why I doubt if efforts like OpenAI will ultimately turn out to be beneficial or catastrophic. I understand their purpose of democratizing the power and benefits of AI technology broadly. However, if their platform can provide a critical component or developmental path to AGI in the future, the unrestrained distribution of the technology is similar to spreading the blueprint of nuclear warheads. It could result in an equivalent of nuclear arms race, but worse--because a small team of capable programmers of any rogue organizations could come up with new improvements. It is true that we are probably quite far from AGI, but nobody really knows how far. We could be closer than most experts think. Few experts in 2006 would expect deep neural networks to beat a Go champion or be able to describe visual scenes in natural language sentences within a decade. Unexpected things happen, and we are much further behind in terms of preventive technology like Friendly AI. (https://en.wikipedia.org/wiki/Friendly_artificial_intelligence https://en.wikipedia.org/wiki/Friendly_artificial_intelligen...) Given the existential risks involved, I would propose more careful handling of the technology. (I agree that opening up most other technologies is usually a positive thing.) See "Superintelligence" for comprehensive arguments against unfettered development of AGI: https://en.wikipedia.org/wiki/Superintelligence:_Paths,_Dangers,_Strategies https://en.wikipedia.org/wiki/Superintelligence:_Paths,_Dang...

markdog12 10y ago
It's a little repetitive and slow in places, but I wish more people read that book. It outlines some big issues with AGI. Changed my outlook on AI for sure.
YeGoblynQueenne 10y ago
>> the unrestrained distribution of the technology is similar to spreading the blueprint of nuclear warheads. There's an episode of Pinky and the Brain where Brain invents a magnification ray and gigantises Pinky and himself - so they're now giant rats [1]. They walk about some human metropolis, looming larger than skyscrapers and terrorising everyone. To cut the story short, Pinky happens and eventually the gun shoots everything in the world. ... so Pinky and the Brain are now normal-sized mice again. Or, well, they're giant mice, but everything in the world is also giant, so they're small. I think of that episode during discussions of the dangers of AGI and how "democratizing" it will eliminate those dangers. The problem is that AGI, like nuclear weapons, is not an equalizing force only- it is also a very destructive force, and so there is the danger that any loon with access to it could blow us all to kingdom come (with nuclear weapons, for sure; with AGI, perhaps). On the other hand, when nuclear weapons, or AGI, giant rays etc are not democratized we have the situation we're in today: only a few entities have access to them and they basically have free (military) reign... and there is still the danger that some loon might get to a position of power were they can push a button and -blamo. Which is why people are really worried about this sort of thing. Because once certain discoveries are made, there's no going back and at the same time the road ahead is full of danger. [1] https://www.youtube.com/watch?v=-fPcqu5aMyw https://www.youtube.com/watch?v=-fPcqu5aMyw

iandanforth 10y ago

Marketing, not confusion.

visarga 10y ago

> The idea that we should invent AGI to make "personal digital assistants" is like Hogwarts students inventing time machines so they can take two classes at once. I disagree. Chatbots and physical robots are where AGI is going. It's basically AI that is interacting with people and the world directly, not just computing away in a cluster, broken from reality.