3 ms·
You don't have assign high probability to hard takeoff to support ethical/friendly AI research. The consequences of a hard takeoff are so huge (potentially dest
by AngryParsley 16y ago
You don't have assign high probability to hard takeoff to support ethical/friendly AI research. The consequences of a hard takeoff are so huge (potentially destroying humanity) that even a low chance is worth making lower.
Humans have been running on the same hardware for several thousand years. Just our improved knowledge and culture has been enough to keep us growing at an exponential rate. If our brains were end-user modifiable and had APIs, we'd be able to increase our growth rate even faster. An AI could do even better than self-modification. It could buy or hack enough computers to give itself thousands of times more processing power.
I don't assign high probability to that sort of scenario, but it is high enough that it's worth ameliorating.