9 ms·
This article is typical of many that claim proven limits on the feasibility (or, in this case, the capabilities) of generalized artificial intelligence, in that
by mannykannot 9y ago
This article is typical of many that claim proven limits on the feasibility (or, in this case, the capabilities) of generalized artificial intelligence, in that it structures the argument in a way designed to avoid discussion of the issue.
It starts by claiming that there is no such thing as general intelligence. What specialized intelligence, then, is human intelligence? It's specialized for "being human". The author is apparently unaware that this tautological response eliminates the distinction between general and specialized intelligence, as one could just as validly (or vacuously) say that a superhuman intelligence is specialized in being what it is and doing what it does. The author has invalidated the hook on which he had hung his argument.
A lot of column-inches are expended on repeatedly restating that animal intelligences have co-evolved with their sensorimotor systems, which is a contingent fact of history, not a fundamental necessity for intelligence (as far as we know; but then the whole article is predicated on the feasibility of AI.) He raises the 'brain in a vat' trope, but no-one is suggesting that AIs must be disconnected from the external world. Furthermore, this line of argument ignores the fact that many of the greatest achievements of human intelligence have come from contemplating abstract ideas.
When the author writes "most of our intelligence is not in our brain, it is externalized as our civilization", he is confusing the achievements of intelligent agents for intelligence itself. When he writes that "an individual brain cannot implement recursive intelligence augmentation" he is confusing a limit on human capabilities for a fundamental limit on intelligence itself...
I am far from convinced that the singularity must follow from achieving human-level artificial intelligence, as we don't know how to get to the starting line, let alone know how the problem of bootstrapping intelligence scales, but the arguments presented here do nothing to persuade me that it is impossible.
- erasemus 9y ago>It starts by claiming that there is no such thing as general intelligence. What specialized intelligence, then, is human intelligence? I think there are two usages of the term 'general intelligence' floating around: (1) the ability that humans possess (but which animals don't) to create universal theories, (2) the measure of one human's general cognitive ability or potential (in all fields) relative to another human's. Note that IQ tests are concerned with (2). The quest for AGI is concerned with (1), though the additional prediction of intelligence explosion or singularity assumes the validity of (2). I think the author would claim that (1) exists but (2) doesn't. He explains the predictive power of IQ tests by claiming that general intelligence is a threshold ability and that people who score highly on an (arbitrary) test are more likely to have exceeded that threshold. Beyond the threshold, achievement is limited only by other factors.
- mannykannot 9y agoThe author seems to want it both ways: he is skeptical that IQ measures intelligence (and certainly not general intelligence, the existence of which he denies), but he uses it in his arguments. As I see it, the author's IQ argument has three claims: that people with exceptional IQ do not generally achieve commensurately that much better than the rest of us, that this is because they are constrained by the state of the intellectual environment they live in, and that there is a threshold such that IQ over that level is unhelpful. This last claim is allegedly a conclusion drawn from its predecessors, but to me, it looks like the first two claims provide a possible explanation for why there might be an apparent threshold effect, regardless of whether there actually is one. Nothing in this passage convincingly denies the notion that a sufficient number of communicating, above-average intelligent agents could expand the intellectual environment faster than we are currently able to achieve.
- visarga 9y agoFaster, yes, not exponentially faster. With a thousand times more scientists you don't improve the speed of discovery 1000x.
- daveguy 9y agoExcellent point. On the contrary. You don't even improve it +1000. If you are lucky you improve it +log(1000).
- erasemus 9y agoI skimmed the article again and didn't find any obvious equivocations regarding IQ and intelligence. But, regardless of this, I think solving problems requires creativity, not intelligence. Creativity seems to be independent of knowing how to do IQ tests. It's also, pace the author, independent of the environment. The main limiting factor is whether you want some particular knowledge. >sufficient number of communicating, above-average intelligent agents could expand the intellectual environment Haven't we already got that, with the internet? Groups tend to be dominated by groupthink which is why creative individuals are aloof. Nevertheless, a young group of AGIs would be fairly isolated from humanity simply by virtue of being non-human. So, though their starting point would be determined by the state of our knowledge at their birth, they may well make rapid progress for a short period, rather like the renaissance or the USA in the late 1800s when the nation was young and expanding. Then presumably they would fall prey to groupthink and pessimism just as most adults and nations do eventually. Progress would slow considerably.
- munificent 9y ago> What specialized intelligence, then, is human intelligence? I think the author just means "the skills needed to pilot a human body on Earth in a normal human social environment". I see no tautology, and the preceding sentence about octopuses makes the author's meaning here pretty clear.
- AstralStorm 9y agoI suppose one has to know calculus and topology to navigate Earth. Worth a smattering of quantum physics and wave physics. Oh wait. Those do not fall under such narrow definition of intelligence.
- jessaustin 9y agoHow many insects or birds know "calculus and topology"? You're straining so much here, to be both narrow and broad, that you contradict yourself.
- AstralStorm 9y agoExactly the point. Those skills are not needed to successfully navigate the Earth, yet we have developed them. The rationale was general self improvement and more control over nature. So the general premise is to ensure better and higher survival. Which is a changing objective function. No reason why a GAI wouldn't have a similar instinct or capabilities. Author also misposes no free lunch theorem. It pertains to algorithms not computational ability, and we're entirely not sure if true intelligence is an immutable algorithm. (Human is not, because of genetics.) There is no real equivalent theorem for self modifying algorithms, (or changing objective functions) and if there were, it will run afoul of halting problem. Three theorem also had at least one known hole where it does not apply to coevolutionary algorithms. (Such as part of Alpha Go to go for an easy example.) While environment puts on a hard limit, the limit is extremely high, on the order of 2^300. The optimising does not have to go for the simplest algorithm to solve the problem. And we do not know enough about informational capacity of matter in quantum states to be honest.
- YeGoblynQueenne 9y ago>> A lot of column-inches are expended on repeatedly restating that animal intelligences have co-evolved with their sensorimotor systems, which is a contingent fact of history, not a fundamental necessity for intelligence (as far as we know; How do we know that? We only know of intelligences that co-evolved with their sensoriotor etc, so how do we know that's not the only way to do it?
- mannykannot 9y ago'As far as we know' is an English-language idiom that expresses a degree of uncertainty.
- visarga 9y agoYour reply gives me the impression that you are not up to date with Reinforcement Learning. If you did, you would know that the author really understands this domain and was not merely tautological. "Specialized at being human" - this is a deep intuition. We are reinforcement learning agents that are pre-programed with a certain number of reward responses. We learn from rewards to keep ourselves alive, to find food, company and make babies. It's all a self reinforcing loop, where intelligence has the role of keeping the body alive, and the body has the role of expressing that intelligence. We're really specialized in keeping human bodies alive and making more human bodies, in our present environment. The author puts a hard limit on intelligence because intelligence is limited by the complexity of the problems it needs to solve (assuming it has sufficient abilities). So the environment is the bottleneck. In that case, an AGI would be like an intelligent human, a little bit better than the rest, not millions of times better.
- mannykannot 9y agoIt has nothing to do with any particular model of learning, let alone of intelligence in general. From the point of view that you have expressed here, it would seem a little surprising that "specialized at being human" includes things like calculus and topology. How do you include these things in a definition of the "being human" speciality that does not turn it into a vacuous category that says the specialization is everything that humans have shown themselves to be capable of, nothing more and nothing less? If it were valid, one could take the argument in your last paragraph to draw a line at any point in the evolution of intelligence and say "this is as good as it gets."
- alfrednachos 9y agoWhat are humans specialized in doing? Because it seems to me that humans are pretty good at chess, calculus, social manipulation, flying to the moon, building machines that take us to the bottom of the ocean, discovering fundamental physics, etc. A fish, no matter what environment and upbringing you give it, can't do any of those things. So it seems like there's some dimension in which the human brain is more generally intelligent than a fish's.
- 9y ago
- doxos 9y ago> The author is apparently unaware that this tautological response eliminates the distinction between general and specialized intelligence, as one could just as validly (or vacuously) say that a superhuman intelligence is specialized in being what it is and doing what it does. It's not a tautology. "Generality" and "specificity" are artifacts of the human experience. What is tautological is to say, "it's general to me, therefore it is general." You think that more and more progress comes by way of more and more optimization. This is not the whole story. Accidentation is the missing ingredient. Humans - as well as all life on earth - have a knack for creating more and more problems. It is this never-ending fountain of new, accidental problems that allows for what appears to us to be a chain of "progress" stretching into the past. Our "generality" is in fact a hairball collection of specific functions that have accreted into the human animal over millions and millions of years. Some abstract Java class called `Agent` with an `.optimize()` method hanging off of it simply does not have that context. If you want a really high quality, generally intelligent function in silicon though, it's hard to beat the XOR function ;)