3 ms·
1- Having children isn't very parallel to designing machines. (It might be more parallel if Hawking's parents specifically had a goal of producing more and more
by schoen 11y ago
1- Having children isn't very parallel to designing machines. (It might be more parallel if Hawking's parents specifically had a goal of producing more and more intelligent human beings, and genetically engineered their children to try to make that happen.) Human reproduction involves a lot of random variation and not a lot of goals, expertise, or the wherewithal to achieve them.
2- I think this is an interesting point about possible inherent computational limits on the ability to solve some problems that we might care about, including in designing more intelligent machines.
3- This is something people have thought about quite a lot. What superintelligent machines do depends on what they've been programmed to do. It's very unlikely that an AI would inherently value "diversity" or "fate" unless it were programmed to do so. The AI wouldn't spontaneously create new values (unless it were programmed to). Most concerns about AIs that exterminate humanity are based in the possibility that an AI would fulfil other goals in a surprising or unanticipated way, with bad side-effects for human survival.
4- Intelligence helps people wage war and dominate others more violently, both by coordinating better to do so (including motivating people to join in), and by developing new technology that helps make larger-scale violence cheaper. Weapons research can help you learn how to kill more people faster and at lower cost. A superintelligent AI could engage in this kind of research if it saw an important reason to.
5- I think that's exactly right; perhaps the important difference here is that the machine version would be more flexible (if you wanted to try overclocking it, or modifying the software somehow). This is dangerous and expensive and confusing to do with a physical brain, because it's hard to manipulate the details of its organization and structure, and because you can just die if you mess up. Think of the ease with which you can edit a PNG or SVG file in a computer compared to editing an oil painting. Perhaps with the computer version you can also run multiple copies in parallel -- something you also can't do with your physical self.