4 ms·

> I.Q. represents the idea that intelligence can be usefully captured by a single number, this idea being one of the assumptions made by proponents of an intell

> I.Q. represents the idea that intelligence can be usefully captured by a single number, this idea being one of the assumptions made by proponents of an intelligence explosion. I think it's not a load bearing assumption for the intelligence explosion hypothesis, but just a premise that allows the notion of "more intelligent than" to be meaningfully expressed, for the sake of having a conversation about this idea. (Comparisons imply some way of ordering things, even if those things are nebulous and multidimensional concepts). > For example, it’s entirely possible that the best that a person with an I.Q. of 300 can do is increase another person’s I.Q. to 200. That would allow one person with an I.Q. of 300 to grant everyone around them an I.Q. of 200, which frankly would be an amazing accomplishment. But that would still leave us at a plateau; there would be no recursive self-improvement and no intelligence explosion. When it's phrased in terms of increasing the intelligence of already-living people, that makes it sound harder than when you phrase it as building a new thing that's better at building than the thing that built it. We can, for example, make machine tools that are faster or more accurate than the machine tools we make them with. I think the premise of an intelligence explosion usually starts with human programmers creating an AI that is at least slightly better than they are themselves, at at least the task of AI programming. If the author subscribes to the refutation that I quoted, then I think it follows that they also wouldn't believe this premise is possible to begin with. Perhaps they'd also be brave enough to say what other things they don't think an AI will ever be better than humans at? Perhaps starting with the least impressive tasks, so we'll find out sooner if the predictions are correct? > This ability of humans to build on one another’s work is precisely why I don’t believe that running a human-equivalent A.I. program for a hundred years in isolation is a good way to produce major breakthroughs. As the saying goes, if I'm going to spend ten hours cutting down trees, I'd spend nine of them sharpening my axe. Just as a society of humans can develop tools to improve their capabilities, so can a single human, as the author's own example of Isaac Newton shows. If the AI is going to be locked in isolation for a hundred years to write software, why doesn't the author think it will spend some of that time on tools to improve its own productivity?

tehjoker 6y ago
I wonder to what extent running a quality AI will require a lot of FLOPs as a baseline or specialized hardware to achieve low FLOPs. Human brains fire only paths in use (and do everything in parallel), but a neural network needs to recalculate the weights on the whole network each training pass for example. Maybe there are some newer techniques that reduce that somewhat.
- wutbrodo 6y ago
  I don't think it's a given that the architectures that get us to AGI will be constrained in exactly the way current neural networks are.
  - CodeGlitch 6y ago
    Yes I could be like the difference between birds and planes, still using the laws of aerodynamics but in a different way. One of those scales, the other doesn't.
ALittleLight 6y ago
Even saying "it's entirely possible" admits the possibility of the alternative. i.e. that a 300 IQ person could invent a way to improve IQs. That seems especially likely if the 300 IQ person came about as a result of efforts led by 100 IQ people. Seems even more plausible to think about like "Could a really smart thing invent computers that are faster/better?"
- aeternum 6y ago
  Even without computers, we've selected for intelligence in animals simply through breeding. Suppose there were a group of monks dedicated to achieving high intellect. One could imagine that they form a pact for many generations that only the most intelligent among them will have children. It's hard to imagine that this wouldn't result in intelligence increases.
web007 6y ago
The first argument on IQ is the weakest of the article. We can't as a human increase IQ inherently outside of something like iodization. We can, however, create faster computers. This is the second or third argument, and here the author gets something fundamentally wrong. We have, for the past decades, used faster computers to help us make faster computers. Their argument is that it's self-limiting, but that's provably false viz current CPU architectures. The one thing the author skirts around that DOES seem to be a strike against the singularity is the asymptotic nature of those kinds of improvements. We can spend N amount of resources for a 100 unit improvement in speed/efficiency/whatnot, but spending 10N resources does not give us 1000 unit improvement, but rather 500 or 150.
simiones 6y ago
> I think the premise of an intelligence explosion usually starts with human programmers creating an AI that is at least slightly better than they are themselves, at at least the task of AI programming. If the author subscribes to the refutation that I quoted, then I think it follows that they also wouldn't believe this premise is possible to begin with. The author is quite explicit about this: he is contradicting that the singularity must be real, not trying to claim that the singularity can't be real. If you agree with his refutation of the arguments for the singularity, you'll be left with the claim "the singularity may or may not happen, we don't know", not with the claim "it's impossible for the singularity to happen". > > This ability of humans to build on one another’s work is precisely why I don’t believe that running a human-equivalent A.I. program for a hundred years in isolation is a good way to produce major breakthroughs. > As the saying goes, if I'm going to spend ten hours cutting down trees, I'd spend nine of them sharpening my axe. Just as a society of humans can develop tools to improve their capabilities, so can a single human, as the author's own example of Isaac Newton shows. If the AI is going to be locked in isolation for a hundred years to write software, why doesn't the author think it will spend some of that time on tools to improve its own productivity? I think a better argument against the author's claim here is that thinking of 1 AI as 1 human is probably wrong. Since an AI can be trivially copied and run independently, it's more useful to think of 1 AI as 1 society.

tehjoker 6y ago

I wonder to what extent running a quality AI will require a lot of FLOPs as a baseline or specialized hardware to achieve low FLOPs. Human brains fire only paths in use (and do everything in parallel), but a neural network needs to recalculate the weights on the whole network each training pass for example. Maybe there are some newer techniques that reduce that somewhat.

wutbrodo 6y ago
I don't think it's a given that the architectures that get us to AGI will be constrained in exactly the way current neural networks are.
- CodeGlitch 6y ago
  Yes I could be like the difference between birds and planes, still using the laws of aerodynamics but in a different way. One of those scales, the other doesn't.

ALittleLight 6y ago

Even saying "it's entirely possible" admits the possibility of the alternative. i.e. that a 300 IQ person could invent a way to improve IQs. That seems especially likely if the 300 IQ person came about as a result of efforts led by 100 IQ people. Seems even more plausible to think about like "Could a really smart thing invent computers that are faster/better?"

aeternum 6y ago
Even without computers, we've selected for intelligence in animals simply through breeding. Suppose there were a group of monks dedicated to achieving high intellect. One could imagine that they form a pact for many generations that only the most intelligent among them will have children. It's hard to imagine that this wouldn't result in intelligence increases.

web007 6y ago

The first argument on IQ is the weakest of the article. We can't as a human increase IQ inherently outside of something like iodization. We can, however, create faster computers. This is the second or third argument, and here the author gets something fundamentally wrong. We have, for the past decades, used faster computers to help us make faster computers. Their argument is that it's self-limiting, but that's provably false viz current CPU architectures. The one thing the author skirts around that DOES seem to be a strike against the singularity is the asymptotic nature of those kinds of improvements. We can spend N amount of resources for a 100 unit improvement in speed/efficiency/whatnot, but spending 10N resources does not give us 1000 unit improvement, but rather 500 or 150.

simiones 6y ago

> I think the premise of an intelligence explosion usually starts with human programmers creating an AI that is at least slightly better than they are themselves, at at least the task of AI programming. If the author subscribes to the refutation that I quoted, then I think it follows that they also wouldn't believe this premise is possible to begin with. The author is quite explicit about this: he is contradicting that the singularity must be real, not trying to claim that the singularity can't be real. If you agree with his refutation of the arguments for the singularity, you'll be left with the claim "the singularity may or may not happen, we don't know", not with the claim "it's impossible for the singularity to happen". > > This ability of humans to build on one another’s work is precisely why I don’t believe that running a human-equivalent A.I. program for a hundred years in isolation is a good way to produce major breakthroughs. > As the saying goes, if I'm going to spend ten hours cutting down trees, I'd spend nine of them sharpening my axe. Just as a society of humans can develop tools to improve their capabilities, so can a single human, as the author's own example of Isaac Newton shows. If the AI is going to be locked in isolation for a hundred years to write software, why doesn't the author think it will spend some of that time on tools to improve its own productivity? I think a better argument against the author's claim here is that thinking of 1 AI as 1 human is probably wrong. Since an AI can be trivially copied and run independently, it's more useful to think of 1 AI as 1 society.