4 ms·

> Second, it is absolutely false that improvement is incremental. Consider the game of Go. For a long time, computers sucked at Go. Then we came up with Monte C

> Second, it is absolutely false that improvement is incremental. Consider the game of Go. For a long time, computers sucked at Go. Then we came up with Monte Carlo search about a decade ago and suddenly we were at strong amateur. Then deep learning was applied, and Alpha-Go jumped to able to beat any human in the world, and has continued to improve. I think you missed his point that this sort of improvement in specific task performance tends to not generalize into improvement at _any_ task. He specifically used AlphaGo as an example of a very narrow specialized problem... i.e, AlphaGo will likely never contribute to solving anything but Go or very closely related tasks. > Third, linear progress is not the history of technology. The history of technology is exponential progress over and over again on everything from how far a steam ship could travel without refueling to the number of operations per second a CPU can carry out. Each expansion of which is counterbalanced by many other inefficiencies (maximum speed of steam ship hulls, carrying capacity, heat losses of faster CPUs and timing/latency problems that put limits on signal propagation, etc). > But on a more relevant level, the task itself is clearly possible. If we have a computer program that is able to design better hardware, it can be improved by simply moving it to better hardware. But it is capable of designing that hardware. This creates a feedback loop which should shorten the cycles in Moore's law, resulting in very few years in a truly superior AI capability. I think you missed the main argument of the whole article, which was basically that every exponential increase in one measure is rooted in the civilization and circumstances as a whole, and therefore never truly reaches runaway effect. Too many other supporting circumstances are needed for any one change to truly dominate the overall system.

btilly 9y ago
I think you missed his point that this sort of improvement in specific task performance tends to not generalize into improvement at _any_ task. He specifically used AlphaGo as an example of a very narrow specialized problem... i.e, AlphaGo will likely never contribute to solving anything but Go or very closely related tasks. Actually he was all over the place, and you jumped from one point that he made to a different one that he also made which I didn't bother addressing. The point that I addressed is that he claimed that improvement is always incremental. It isn't. The point that you brought up is that improvements are narrow, with AlphaGo as an example. This is a horrible example that undermines its point. AlphaGo is the result of applying deep learning techniques to the problem of Go. But as https://en.wikipedia.org/wiki/Deep_learning https://en.wikipedia.org/wiki/Deep_learning indicates, that technique has created huge jumps in the abilities of computers at things as different as Go, image processing, speech recognition, and customer recommendations. In other words the advance here wasn't narrow and targeted. It was broad and widely applicable. Which brings us to a fairly fundamental fact. Human brains have a specific set of capabilities which are each fairly well localized in the brain. Put them together, and they create a very good generalized machine. We are currently able to build affordable computing systems with raw computational power that is roughly comparable to a human brain. (The way that they work is not apples to apples.) But we are unable to build all of those capabilities. And we can't tie it together. However what that says is that we have a limited number of software problems to solve, and then we'll be able to build AI to match humans. Those software problems are open ended, they might be 5 or 50 years away. We don't know. But our brains represent a proof that the problem can be solved, and eventually we'll figure it out. Then what happens next? Each expansion of which is counterbalanced by many other inefficiencies (maximum speed of steam ship hulls, carrying capacity, heat losses of faster CPUs and timing/latency problems that put limits on signal propagation, etc). Not according to the history of technology as I understand it. For example if you look at the history of steam ships in the 1800s you see exponential curves in range and carrying capacity going together, with significant increases in speed as well. If you look at computer systems what you see is different curves resulting in changing bottlenecks. For example CPU speed and hard drive capacity both increase faster than bandwidth/latency between the CPU and the hard drive, making disk latency a growing problem relative to the rest of the system. In absolute numbers, though, a recent computer is better in all dimensions than one from 20 years ago, and better than one from 10 years ago in all dimensions except clock speed. I think you missed the main argument of the whole article, which was basically that every exponential increase in one measure is rooted in the civilization and circumstances as a whole, and therefore never truly reaches runaway effect. Too many other supporting circumstances are needed for any one change to truly dominate the overall system. The growing impact of computers from 1950 to the present sure looks like a runaway effect to me. If strong AI develops, the curve will remain exponential but should steepen abruptly. While it technically won't be a singularity, improvements will come faster than humans can keep track of.

btilly 9y ago

I think you missed his point that this sort of improvement in specific task performance tends to not generalize into improvement at _any_ task. He specifically used AlphaGo as an example of a very narrow specialized problem... i.e, AlphaGo will likely never contribute to solving anything but Go or very closely related tasks. Actually he was all over the place, and you jumped from one point that he made to a different one that he also made which I didn't bother addressing. The point that I addressed is that he claimed that improvement is always incremental. It isn't. The point that you brought up is that improvements are narrow, with AlphaGo as an example. This is a horrible example that undermines its point. AlphaGo is the result of applying deep learning techniques to the problem of Go. But as https://en.wikipedia.org/wiki/Deep_learning https://en.wikipedia.org/wiki/Deep_learning indicates, that technique has created huge jumps in the abilities of computers at things as different as Go, image processing, speech recognition, and customer recommendations. In other words the advance here wasn't narrow and targeted. It was broad and widely applicable. Which brings us to a fairly fundamental fact. Human brains have a specific set of capabilities which are each fairly well localized in the brain. Put them together, and they create a very good generalized machine. We are currently able to build affordable computing systems with raw computational power that is roughly comparable to a human brain. (The way that they work is not apples to apples.) But we are unable to build all of those capabilities. And we can't tie it together. However what that says is that we have a limited number of software problems to solve, and then we'll be able to build AI to match humans. Those software problems are open ended, they might be 5 or 50 years away. We don't know. But our brains represent a proof that the problem can be solved, and eventually we'll figure it out. Then what happens next? Each expansion of which is counterbalanced by many other inefficiencies (maximum speed of steam ship hulls, carrying capacity, heat losses of faster CPUs and timing/latency problems that put limits on signal propagation, etc). Not according to the history of technology as I understand it. For example if you look at the history of steam ships in the 1800s you see exponential curves in range and carrying capacity going together, with significant increases in speed as well. If you look at computer systems what you see is different curves resulting in changing bottlenecks. For example CPU speed and hard drive capacity both increase faster than bandwidth/latency between the CPU and the hard drive, making disk latency a growing problem relative to the rest of the system. In absolute numbers, though, a recent computer is better in all dimensions than one from 20 years ago, and better than one from 10 years ago in all dimensions except clock speed. I think you missed the main argument of the whole article, which was basically that every exponential increase in one measure is rooted in the civilization and circumstances as a whole, and therefore never truly reaches runaway effect. Too many other supporting circumstances are needed for any one change to truly dominate the overall system. The growing impact of computers from 1950 to the present sure looks like a runaway effect to me. If strong AI develops, the curve will remain exponential but should steepen abruptly. While it technically won't be a singularity, improvements will come faster than humans can keep track of.