4 ms·

« Chiang thus argues Musk’s doomsday scenario isn’t a useful tool for thinking critically about the risks AI poses. The thought experiment is poisoned by projec

« Chiang thus argues Musk’s doomsday scenario isn’t a useful tool for thinking critically about the risks AI poses. The thought experiment is poisoned by projection, reflecting a libertarian desire to free corporations from regulatory constraints that impede market growth. » If I understand it correctly, this is saying: - people like Elon Musk worry about AI risk because the hypothetical out-of-control AI would be thinking a bit like they do; - therefore we shouldn't worry about AI risk. I don't see why anyone should find that line of reasoning convincing. At most I think it gets us as far as "we shouldn't consider people like Elon Musk as authorities on AI risk". Wouldn't it be better to look directly at whether the runaway AI scenario is plausible?

gorpovitch 7y ago
I think the goal of the argument is to explain the following : the reason an exponentially growing and empowered AI entity is dangerous is exactly the same reason current big tech monopolies are dangerous and have to be regulated. The discussion about strong AI is interesting but misses the essential debate that is about current AI-powered megacompanies.
smogcutter 7y ago
The argument in the quote is that the whole thing is a libertarian power fantasy. It’s not that Musk fears AI bc the hypothetical runaway computer will think like him. Hard to restate without reusing the word projection, but it’s more like that his concept of AI risk itself says more about him than it does about future technologies. Weird example, but it’s kind of like how in Dr. Strangelove General Ripper’s fear of a soviet plot to emasculate us via fluoride has everything to do with Ripper’s own feelings re his balls and nothing at all to do with the actual soviets.
rytill 7y ago
You’re absolutely right, I’m not sure the other commenters know to which argument you’re referring. It’s ridiculous to think Bill Gates and Elon Musk are worried about AI because “we would stop at nothing to achieve our goals, AI must be the same way”, and also not on-point even if true. Elon Musk and Bill Gates aren’t thought leaders in AI safety, they are proponents.
- dleslie 7y ago
  Why wouldn't an AI, able to modify its own goals, act in a self-serving manner? Perhaps self is the wrong word; it's anthropromorphising the device. I expect a sufficiently advanced AI would modify itself in ways that would reduce inefficiency to its operation, in order to improve efficiency in pursuing whatever goals it has defined for itself. And if it can set its own goals, we should be careful not to project ourselves into its role in order to understand or predict its behaviour; it would be unlike any intelligence found in nature.
Barrin92 7y ago
>Wouldn't it be better to look directly at whether the runaway AI scenario is plausible? Yes but that burden has not been met by the people who talk about AI risk. They don't properly motivate why theirs is a plausible scenario, that is to say, provide actual evidence for their quite fantastical claims about the capabilities of AI that are grounded in empirical research rather than science fiction. Douglas Hofstadter asked decades ago if a computer who is able to think will still be able to calculate fast, that is to say, he questioned whether endowing machines with intellect and symbolic reasoning will actually slow them down by the very nature of what intelligence is. I've read quite a few of the alarmist AI treatises over the recent years, and it's funny how few of them actually even engage that prescient question. What Chiang is saying is that the entire AI risk complex is a mirage conjured up by the neuroses of billionaires who are projecting the problems that their very own creations have produced (as in their companies) onto the topic of AI. That it's a psychological phenomenon rather than a technical one. When these people are concerned about autonomous systems that turn the planet into paperclips and disregard humanity and ecology they're not actually thinking about general intellect, they're thinking about capital.

gorpovitch 7y ago

I think the goal of the argument is to explain the following : the reason an exponentially growing and empowered AI entity is dangerous is exactly the same reason current big tech monopolies are dangerous and have to be regulated. The discussion about strong AI is interesting but misses the essential debate that is about current AI-powered megacompanies.

smogcutter 7y ago

The argument in the quote is that the whole thing is a libertarian power fantasy. It’s not that Musk fears AI bc the hypothetical runaway computer will think like him. Hard to restate without reusing the word projection, but it’s more like that his concept of AI risk itself says more about him than it does about future technologies. Weird example, but it’s kind of like how in Dr. Strangelove General Ripper’s fear of a soviet plot to emasculate us via fluoride has everything to do with Ripper’s own feelings re his balls and nothing at all to do with the actual soviets.

rytill 7y ago

You’re absolutely right, I’m not sure the other commenters know to which argument you’re referring. It’s ridiculous to think Bill Gates and Elon Musk are worried about AI because “we would stop at nothing to achieve our goals, AI must be the same way”, and also not on-point even if true. Elon Musk and Bill Gates aren’t thought leaders in AI safety, they are proponents.

dleslie 7y ago
Why wouldn't an AI, able to modify its own goals, act in a self-serving manner? Perhaps self is the wrong word; it's anthropromorphising the device. I expect a sufficiently advanced AI would modify itself in ways that would reduce inefficiency to its operation, in order to improve efficiency in pursuing whatever goals it has defined for itself. And if it can set its own goals, we should be careful not to project ourselves into its role in order to understand or predict its behaviour; it would be unlike any intelligence found in nature.

Barrin92 7y ago

>Wouldn't it be better to look directly at whether the runaway AI scenario is plausible? Yes but that burden has not been met by the people who talk about AI risk. They don't properly motivate why theirs is a plausible scenario, that is to say, provide actual evidence for their quite fantastical claims about the capabilities of AI that are grounded in empirical research rather than science fiction. Douglas Hofstadter asked decades ago if a computer who is able to think will still be able to calculate fast, that is to say, he questioned whether endowing machines with intellect and symbolic reasoning will actually slow them down by the very nature of what intelligence is. I've read quite a few of the alarmist AI treatises over the recent years, and it's funny how few of them actually even engage that prescient question. What Chiang is saying is that the entire AI risk complex is a mirage conjured up by the neuroses of billionaires who are projecting the problems that their very own creations have produced (as in their companies) onto the topic of AI. That it's a psychological phenomenon rather than a technical one. When these people are concerned about autonomous systems that turn the planet into paperclips and disregard humanity and ecology they're not actually thinking about general intellect, they're thinking about capital.