3 ms·

Interesting and quite dense post. Singularity related articles and worries about "good" AI building makes me a bit uneasy when the performance of machines on ma

by Jun8 15y ago

Interesting and quite dense post. Singularity related articles and worries about "good" AI building makes me a bit uneasy when the performance of machines on many important tasks, e.g. object recognition, is dismal. So we have quite a way to go. The usual rebuttal to the above is the "intelligence explosion" argument: once AIs can improve themselves, their progress will be exponential. As a tl;dr for the post, consider the Dewey summary of the argument: 1. Hardware and software are improving, there are no signs that we will stop this, 2. and human biology and biases indicate that we are far below the upper limit on intelligence. 3. Economic arguments indicate that most AIs would act to become more intelligent. 4. Therefore, intelligence explosion is very likely. If you think about it, only (1) is a fact, the rest are assertions that are open to debate. For example, in (2) how do we define the "upper limits of intelligence", when even defining intelligence is problematic. Currently the human brain is the most complex and intelligent object we know. For (3), I don't know what sort of "economic arguments" are meant but as we all know becoming "more intelligent" is not a simple hill climbing process, in fact it's not clear how to go about it. Is Watson more intelligent than a person? If you think about these facts for some time, you will find that AIs designing more intelligent AIs scenario has very little plausibility actually, let alone being highly probable.

dncrane 15y ago
"Singularity related articles and worries about "good" AI building makes me a bit uneasy when the performance of machines on many important tasks, e.g. object recognition, is dismal. So we have quite a way to go." Yudkowsky (2008), "AI as a Negative and Positive Factor in Global Risk" http://singinst.org/upload/artificial-intelligence-risk.pdf http://singinst.org/upload/artificial-intelligence-risk.pdf addresses this in section 7 (though it's really worth it to read the whole paper): "The first moral is that confusing the speed of AI research with the speed of a real AI once built is like confusing the speed of physics research with the speed of nuclear reactions. It mixes up the map with the territory." =============== "in (2) how do we define the "upper limits of intelligence", when even defining intelligence is problematic." Given the huge number of reproducible cognitive biases humans are known to exhibit (http://wiki.lesswrong.com/wiki/Bias http://wiki.lesswrong.com/wiki/Bias), it seems very unlikely that humans are optimally intelligent (one way to define intelligence, from the Omohundro paper below: 'We define “intelligent systems” to be those which choose their own actions to achieve specified goals using limited resources' so "more intelligent" means better at achieving those goals using limited resources) =============== "(3), I don't know what sort of "economic arguments" are meant but as we all know becoming "more intelligent" is not a simple hill climbing process, in fact it's not clear how to go about it." Omohundro (2011), "Rationally Shaped Artificial Intelligence" Makes the case for #3: http://selfawaresystems.files.wordpress.com/2011/10/rationally_shaped_ai.pdf http://selfawaresystems.files.wordpress.com/2011/10/rational... (summary: by "more intelligent" we mean "more capable of achieving goals." So any AI which has goals will act to become more intelligent so that it can more effectively achieve those goals)
- Jun8 15y ago
  Thanks for a great reply! I think my (and I don't think I'm alone in this) bias is that when I think of intelligence I tend to think of tasks that the human brain has evolved to perform, like linguistic and visual analysis. It may be argued that these tasks form a small subspace of the "intelligence space", but, boy, is the human brain good at solving these! That doesn't mean that we are optimal; however, for many AI tasks the brain operates on such a vastly different performance level that comparisons with machine AI seem out of place. Even systems like Watson, which were created by many hundreds of man-years, can stump humans in narrow domain tasks. Now, it might be argued that some sort of paradigm shift will occur as machines get more intelligent (e.g. similar to how results from STR and quantum theory were impossible to predict to Newtonian-constrained thinking) and some effects that we cannot predict now will prevail, like the massive increase in intelligence in humans that caused/was caused by the advent of language.

by Jun8 15y ago

dncrane 15y ago

"Singularity related articles and worries about "good" AI building makes me a bit uneasy when the performance of machines on many important tasks, e.g. object recognition, is dismal. So we have quite a way to go." Yudkowsky (2008), "AI as a Negative and Positive Factor in Global Risk" http://singinst.org/upload/artificial-intelligence-risk.pdf http://singinst.org/upload/artificial-intelligence-risk.pdf addresses this in section 7 (though it's really worth it to read the whole paper): "The first moral is that confusing the speed of AI research with the speed of a real AI once built is like confusing the speed of physics research with the speed of nuclear reactions. It mixes up the map with the territory." =============== "in (2) how do we define the "upper limits of intelligence", when even defining intelligence is problematic." Given the huge number of reproducible cognitive biases humans are known to exhibit (http://wiki.lesswrong.com/wiki/Bias http://wiki.lesswrong.com/wiki/Bias), it seems very unlikely that humans are optimally intelligent (one way to define intelligence, from the Omohundro paper below: 'We define “intelligent systems” to be those which choose their own actions to achieve specified goals using limited resources' so "more intelligent" means better at achieving those goals using limited resources) =============== "(3), I don't know what sort of "economic arguments" are meant but as we all know becoming "more intelligent" is not a simple hill climbing process, in fact it's not clear how to go about it." Omohundro (2011), "Rationally Shaped Artificial Intelligence" Makes the case for #3: http://selfawaresystems.files.wordpress.com/2011/10/rationally_shaped_ai.pdf http://selfawaresystems.files.wordpress.com/2011/10/rational... (summary: by "more intelligent" we mean "more capable of achieving goals." So any AI which has goals will act to become more intelligent so that it can more effectively achieve those goals)

Jun8 15y ago
Thanks for a great reply! I think my (and I don't think I'm alone in this) bias is that when I think of intelligence I tend to think of tasks that the human brain has evolved to perform, like linguistic and visual analysis. It may be argued that these tasks form a small subspace of the "intelligence space", but, boy, is the human brain good at solving these! That doesn't mean that we are optimal; however, for many AI tasks the brain operates on such a vastly different performance level that comparisons with machine AI seem out of place. Even systems like Watson, which were created by many hundreds of man-years, can stump humans in narrow domain tasks. Now, it might be argued that some sort of paradigm shift will occur as machines get more intelligent (e.g. similar to how results from STR and quantum theory were impossible to predict to Newtonian-constrained thinking) and some effects that we cannot predict now will prevail, like the massive increase in intelligence in humans that caused/was caused by the advent of language.