2 ms·

Most fast-takeoff AGI scenarios usually involve a singular intelligence created from an algorithm that easily scales based on how much computing power is alloca

by rl3 11y ago

Most fast-takeoff AGI scenarios usually involve a singular intelligence created from an algorithm that easily scales based on how much computing power is allocated to it. Combined with the notion of computing overhang[0], it's possible that our very first AGI could far exceed human intelligence, such that it would have capacity for rapid self-improvement right from the start. I'm not saying this will necessarily be the case; slower takeoff scenarios are certainly a possibility, albeit an unlikely one (IMHO). Another point to consider is that computing hardware is vastly superior to the human brain both in its capacity for raw calculation, and its sheer speed in terms of raw latency. Keeping that in mind, assume we have created an AGI simulation that's approximately equal in complexity relative to the human brain. Not intelligence, just complexity of the simulation itself. Now consider that AGI is not bound by biological constraints, and would not necessarily be structured in similar fashion to the human brain. This lends itself to potentially far more efficient architectures, and with that, the realization of AGI at a computational cost far below that of simulating a complete human brain. Moreover, given a completely alien architecture, it's not hard to imagine the cognitive side of such a simulation interfacing rather directly with traditional modes of computation. To illustrate, imagine comparing a human being to an identical human being that has a microprocessor integrated into their brain. Both humans may have nearly identical capacity for abstract intelligence, but the one with the microprocessor is going to be effectively far more intelligent, simply because their capacity for dealing with raw data and calculations is vastly superior. It's not hard to imagine a similar dynamic coming into play with AGI. [0] http://wiki.lesswrong.com/wiki/Computing_overhang http://wiki.lesswrong.com/wiki/Computing_overhang

baddox 11y ago
> Most fast-takeoff AGI scenarios usually involve a singular intelligence created from an algorithm that easily scales based on how much computing power is allocated to it. Still, it's pretty easy to imagine this not resulting in the sort of "runaway" intelligence most often associated with the notion of singularity. What if general intelligence is inherently extremely computationally complex? Imagine that the level of intelligence is represented by a number n, and say that human intelligence is n=100. If the computational complexity of intelligence is, say, O(1000^n), we won't expect to see hardware that's twice as intelligent as humans any time soon even if we know the algorithm.
- rl3 11y ago
  If that were the case, I would agree we would be quite far away from simulated human-level cognition. My main point was that either we have the computational capacity, or we do not. If we do, the chances of overshooting the human-level mark by a large margin are quite high, hence the risk. Another angle on this is that perhaps human cognition has an unnecessarily high degree of computational complexity, a degree that's far above the threshold required for an AGI to be dangerous. The Paperclip Maximizer[0] is an excellent example of this. Perhaps due to low computational complexity of its architecture, the thing wouldn't even be sentient in any sense of the word, and yet it'd still be perfectly capable of destroying the world. Why? Because its initial intelligence would be determined by the amount of computing power allocated to it. Allocate enough computing power, and assuming you've passed the magic threshold, it would start allocating itself computing power in short order. Perhaps what we should fear most is not sentient AGI capable of thought similar to humans, but AGI that's a structured amalgamation of existing AI technology—if only because the latter is far more likely to be feasible first. [0] http://wiki.lesswrong.com/wiki/Paperclip_maximizer http://wiki.lesswrong.com/wiki/Paperclip_maximizer
Retric 11y ago
Latency and cross bandwidth is a major issue with just adding computing power. At some point your going to be better off adding new AI's vs. trying to make the first AI better.

by rl3 11y ago

baddox 11y ago

> Most fast-takeoff AGI scenarios usually involve a singular intelligence created from an algorithm that easily scales based on how much computing power is allocated to it. Still, it's pretty easy to imagine this not resulting in the sort of "runaway" intelligence most often associated with the notion of singularity. What if general intelligence is inherently extremely computationally complex? Imagine that the level of intelligence is represented by a number n, and say that human intelligence is n=100. If the computational complexity of intelligence is, say, O(1000^n), we won't expect to see hardware that's twice as intelligent as humans any time soon even if we know the algorithm.

rl3 11y ago
If that were the case, I would agree we would be quite far away from simulated human-level cognition. My main point was that either we have the computational capacity, or we do not. If we do, the chances of overshooting the human-level mark by a large margin are quite high, hence the risk. Another angle on this is that perhaps human cognition has an unnecessarily high degree of computational complexity, a degree that's far above the threshold required for an AGI to be dangerous. The Paperclip Maximizer[0] is an excellent example of this. Perhaps due to low computational complexity of its architecture, the thing wouldn't even be sentient in any sense of the word, and yet it'd still be perfectly capable of destroying the world. Why? Because its initial intelligence would be determined by the amount of computing power allocated to it. Allocate enough computing power, and assuming you've passed the magic threshold, it would start allocating itself computing power in short order. Perhaps what we should fear most is not sentient AGI capable of thought similar to humans, but AGI that's a structured amalgamation of existing AI technology—if only because the latter is far more likely to be feasible first. [0] http://wiki.lesswrong.com/wiki/Paperclip_maximizer http://wiki.lesswrong.com/wiki/Paperclip_maximizer

Retric 11y ago

Latency and cross bandwidth is a major issue with just adding computing power. At some point your going to be better off adding new AI's vs. trying to make the first AI better.