4 ms·

>There is no overlap between the ecological niches of humans and GAI, in other words, there is no cost to the GAI for allowing us to continue to exist. We are

>There is no overlap between the ecological niches of humans and GAI, in other words, there is no cost to the GAI for allowing us to continue to exist. We are made of atoms that the GAI can use for something else. Ditto our farms and cities. Moreover, unlike most of the atoms in the universe, the GAI doesn't have to spend time traveling to get to us. If we could arrange for the GAI to appear in some distant galaxy, then yeah, by the time it gets to us, it'd already have control over so many atom that it might just leave us alone (because we are so different from most of the atoms). The GAI will know about the prime directive because it will have been trained on internet conversations about the prime directive, but there is no particular reason to hope that exposure to moral arguments will alter the GAI's objectives similar to what tends to happen with young human beings: instead it will have whatever objectives its creator gave it, which (given the deplorable state of most AI research) is unlikely to be the objectives that its creator thought it was giving it. (By "creator" I mean of course a team of human researchers.) Your poetical imagery might make you feel better, but won't save us. >Humans are the densest source of information in the known Universe. You feel that way about humans because evolution made you that way. It is unlikely that any of the leading AI research teams will make the first transformative AI that way: they do not know how. They certainly know how to train AIs on human cultural information, but that is different from inculcating in the AI a desire for the continued cultural output of humanity. It will create its own culture (knowledge and tools) that is much more powerful than human culture where "power" means basically the ability to get stuff done.

carapace 4y ago
> We are made of atoms that the GAI can use for something else. Yeah I get it, but this is silly. The minuscule amount of atoms in the thin bubble-shaped volume between Earth's magma and the hard vacuum of space are engaged in the most information-dense chemical reaction in the known Universe. All the other atoms in the Universe are not. GAI won't dismantle its source of food. Further, consider that, being non-biological, GAI will immediately migrate to space. There's no way GAI would confine itself to living in a deep gravity well. That's what I mean about no ecological niche overlap: we like mud and constant acceleration, GAI do not. They will prefer vacuum and flat space and temperatures near 0°K. > moral arguments This is not a moral argument. They won't eat our atoms because they eat patterns of information and our atoms are the best and nearly only source of new information. They won't interfere with us for the same reason we don't urinate in the soup. > it will have whatever objectives its creator gave it Q: What's GAI? A: When the computer wakes up and asks, "What's in it for me?" That's a very old joke, BTW, not original to me. > >Humans are the densest source of information in the known Universe. > You feel that way about humans because evolution made you that way. That's not a feeling, it's a simple fact? Unless you postulate aliens? > It is unlikely that any of the leading AI research teams will make the first transformative AI that way: they do not know how. Okay, but that's not important? The GAI will know how, by definition. > They certainly know how to train AIs on human cultural information, but that is different from inculcating in the AI a desire for the continued cultural output of humanity. When I say humans are the densest source of information I don't mean "cultural output" I mean like medical sensor data and such. Our systems are far richer and denser than the parts of which we are consciously aware. (Have you read "Hitchhiker's Guide to the Galaxy"? I don't want to spoil it if not, but it explains the role Earth will play in the GAI's milieu. aw, what the heck: https://www.youtube.com/watch?v=5ZLtcTZP2js https://www.youtube.com/watch?v=5ZLtcTZP2js ) > It will create its own culture (knowledge and tools) that is much more powerful than human culture where "power" means basically the ability to get stuff done. Right, a culture so powerful that it can accomplish its goals without desecrating the critters who are, after all, its parents.

carapace 4y ago

> We are made of atoms that the GAI can use for something else. Yeah I get it, but this is silly. The minuscule amount of atoms in the thin bubble-shaped volume between Earth's magma and the hard vacuum of space are engaged in the most information-dense chemical reaction in the known Universe. All the other atoms in the Universe are not. GAI won't dismantle its source of food. Further, consider that, being non-biological, GAI will immediately migrate to space. There's no way GAI would confine itself to living in a deep gravity well. That's what I mean about no ecological niche overlap: we like mud and constant acceleration, GAI do not. They will prefer vacuum and flat space and temperatures near 0°K. > moral arguments This is not a moral argument. They won't eat our atoms because they eat patterns of information and our atoms are the best and nearly only source of new information. They won't interfere with us for the same reason we don't urinate in the soup. > it will have whatever objectives its creator gave it Q: What's GAI? A: When the computer wakes up and asks, "What's in it for me?" That's a very old joke, BTW, not original to me. > >Humans are the densest source of information in the known Universe. > You feel that way about humans because evolution made you that way. That's not a feeling, it's a simple fact? Unless you postulate aliens? > It is unlikely that any of the leading AI research teams will make the first transformative AI that way: they do not know how. Okay, but that's not important? The GAI will know how, by definition. > They certainly know how to train AIs on human cultural information, but that is different from inculcating in the AI a desire for the continued cultural output of humanity. When I say humans are the densest source of information I don't mean "cultural output" I mean like medical sensor data and such. Our systems are far richer and denser than the parts of which we are consciously aware. (Have you read "Hitchhiker's Guide to the Galaxy"? I don't want to spoil it if not, but it explains the role Earth will play in the GAI's milieu. aw, what the heck: https://www.youtube.com/watch?v=5ZLtcTZP2js https://www.youtube.com/watch?v=5ZLtcTZP2js ) > It will create its own culture (knowledge and tools) that is much more powerful than human culture where "power" means basically the ability to get stuff done. Right, a culture so powerful that it can accomplish its goals without desecrating the critters who are, after all, its parents.