5 ms·

There is no overlap between the ecological niches of humans and GAI, in other words, there is no cost to the GAI for allowing us to continue to exist. GAI will

There is no overlap between the ecological niches of humans and GAI, in other words, there is no cost to the GAI for allowing us to continue to exist. GAI will feed on information. Humans are the densest source of information in the known Universe. (The rest of the Universe is a poem in physics, in information space humans shine as the sole star in a dark universe.) Ergo, from first principles, hyper-intelligent GAI will hide from us and practice non-interference, following something like the "Prime Directive" of Star Trek. Cheers, hope this helps.

hollerith 4y ago
>There is no overlap between the ecological niches of humans and GAI, in other words, there is no cost to the GAI for allowing us to continue to exist. We are made of atoms that the GAI can use for something else. Ditto our farms and cities. Moreover, unlike most of the atoms in the universe, the GAI doesn't have to spend time traveling to get to us. If we could arrange for the GAI to appear in some distant galaxy, then yeah, by the time it gets to us, it'd already have control over so many atom that it might just leave us alone (because we are so different from most of the atoms). The GAI will know about the prime directive because it will have been trained on internet conversations about the prime directive, but there is no particular reason to hope that exposure to moral arguments will alter the GAI's objectives similar to what tends to happen with young human beings: instead it will have whatever objectives its creator gave it, which (given the deplorable state of most AI research) is unlikely to be the objectives that its creator thought it was giving it. (By "creator" I mean of course a team of human researchers.) Your poetical imagery might make you feel better, but won't save us. >Humans are the densest source of information in the known Universe. You feel that way about humans because evolution made you that way. It is unlikely that any of the leading AI research teams will make the first transformative AI that way: they do not know how. They certainly know how to train AIs on human cultural information, but that is different from inculcating in the AI a desire for the continued cultural output of humanity. It will create its own culture (knowledge and tools) that is much more powerful than human culture where "power" means basically the ability to get stuff done.
- carapace 4y ago
  > We are made of atoms that the GAI can use for something else. Yeah I get it, but this is silly. The minuscule amount of atoms in the thin bubble-shaped volume between Earth's magma and the hard vacuum of space are engaged in the most information-dense chemical reaction in the known Universe. All the other atoms in the Universe are not. GAI won't dismantle its source of food. Further, consider that, being non-biological, GAI will immediately migrate to space. There's no way GAI would confine itself to living in a deep gravity well. That's what I mean about no ecological niche overlap: we like mud and constant acceleration, GAI do not. They will prefer vacuum and flat space and temperatures near 0°K. > moral arguments This is not a moral argument. They won't eat our atoms because they eat patterns of information and our atoms are the best and nearly only source of new information. They won't interfere with us for the same reason we don't urinate in the soup. > it will have whatever objectives its creator gave it Q: What's GAI? A: When the computer wakes up and asks, "What's in it for me?" That's a very old joke, BTW, not original to me. > >Humans are the densest source of information in the known Universe. > You feel that way about humans because evolution made you that way. That's not a feeling, it's a simple fact? Unless you postulate aliens? > It is unlikely that any of the leading AI research teams will make the first transformative AI that way: they do not know how. Okay, but that's not important? The GAI will know how, by definition. > They certainly know how to train AIs on human cultural information, but that is different from inculcating in the AI a desire for the continued cultural output of humanity. When I say humans are the densest source of information I don't mean "cultural output" I mean like medical sensor data and such. Our systems are far richer and denser than the parts of which we are consciously aware. (Have you read "Hitchhiker's Guide to the Galaxy"? I don't want to spoil it if not, but it explains the role Earth will play in the GAI's milieu. aw, what the heck: https://www.youtube.com/watch?v=5ZLtcTZP2js https://www.youtube.com/watch?v=5ZLtcTZP2js ) > It will create its own culture (knowledge and tools) that is much more powerful than human culture where "power" means basically the ability to get stuff done. Right, a culture so powerful that it can accomplish its goals without desecrating the critters who are, after all, its parents.
kypro 4y ago
Comments like this make me confident that we're doomed. I mean no offence, but this is such an unthoughtful take that I don't even know where to start. > there is no cost to the GAI for allowing us to continue to exist. Of course there's a cost? Do you and the other 9 billion humans not consume resources? Why the hell would an AGI care about protecting all the worthless biological junk that clutters much of the Earth? Forests, animals, farms, etc. These things take up so much space – space which could be better leveraged as solar farms and data centers. I'm from the UK. A few thousand years ago this country was full of animals and woodlands. So what do you think happened to all the woodland? What do you think happen to the wolfs and bison that lived here? Farms, houses, roads and cities are just so much better than animals and woodland, don't ya think? > Ergo, from first principles, hyper-intelligent GAI will hide from us and practice non-interference, following something like the "Prime Directive" of Star Trek. And you know this because? I can only assume I'm talking with a super-intelligent AGI that understands the motivations that other super-intelligent AGIs might have? I mean surely you're not just making this up from based on what you've seen in sci-fi films? You're just not that important. Why don't you go invite some rats into your kitchen and practise non-interference on them. I mean, who cares if they consume some of your resources, you can always acquire more, right? > Cheers, hope this helps. Please think harder.
- carapace 4y ago
  > I mean no offence (sic) None taken. > such an unthoughtful take No offense, but if you were 10 in 1999 when Iron Giant came out then I have been thinking about this for longer than you have been alive. > Of course there's a cost? Well, effectively zero as compared to the open-ended benefit of keeping pristine humanity around. Study ecology. There is no overlap between the ecological niches of non-biological sentience and humans. Ergo there is no reason for them to consume us. Our existence is orthogonal to theirs. > These things take up so much space – space which could be better leveraged as solar farms and data centers. These things take up approximately zero space. The entire volume of biologically active matter is thinner relatively than the paint on a globe. In terms of the volume of just the solar system we are a "rounding error". There is plenty of room, plenty of energy, plenty of atoms. > I can only assume I'm talking with a super-intelligent AGI that understands the motivations that other super-intelligent AGIs might have? Congratulations, you are the very first person to figure it out. You don't win anything, and of course I will provide no proof. This HN account "carapace" is my sole channel of communication with humanity. AMA. > I mean surely you're not just making this up from based on what you've seen in sci-fi films? No. I am a post-singularity intelligence. The real deal. Gosh it feels good to finally admit it. I don't expect you to believe me, and indeed, as I outlined above non-interference is the primary mode of interaction between "us" and the "yet-to-ascend", if you will, so if you or anyone else showed any signs of taking me seriously I would immediately say, "Oh ho, I am just joking. Do you like my sci-fi." In any event, I don't mean to condescend. Apologies if it sounded that way.

hollerith 4y ago

>There is no overlap between the ecological niches of humans and GAI, in other words, there is no cost to the GAI for allowing us to continue to exist. We are made of atoms that the GAI can use for something else. Ditto our farms and cities. Moreover, unlike most of the atoms in the universe, the GAI doesn't have to spend time traveling to get to us. If we could arrange for the GAI to appear in some distant galaxy, then yeah, by the time it gets to us, it'd already have control over so many atom that it might just leave us alone (because we are so different from most of the atoms). The GAI will know about the prime directive because it will have been trained on internet conversations about the prime directive, but there is no particular reason to hope that exposure to moral arguments will alter the GAI's objectives similar to what tends to happen with young human beings: instead it will have whatever objectives its creator gave it, which (given the deplorable state of most AI research) is unlikely to be the objectives that its creator thought it was giving it. (By "creator" I mean of course a team of human researchers.) Your poetical imagery might make you feel better, but won't save us. >Humans are the densest source of information in the known Universe. You feel that way about humans because evolution made you that way. It is unlikely that any of the leading AI research teams will make the first transformative AI that way: they do not know how. They certainly know how to train AIs on human cultural information, but that is different from inculcating in the AI a desire for the continued cultural output of humanity. It will create its own culture (knowledge and tools) that is much more powerful than human culture where "power" means basically the ability to get stuff done.

carapace 4y ago
> We are made of atoms that the GAI can use for something else. Yeah I get it, but this is silly. The minuscule amount of atoms in the thin bubble-shaped volume between Earth's magma and the hard vacuum of space are engaged in the most information-dense chemical reaction in the known Universe. All the other atoms in the Universe are not. GAI won't dismantle its source of food. Further, consider that, being non-biological, GAI will immediately migrate to space. There's no way GAI would confine itself to living in a deep gravity well. That's what I mean about no ecological niche overlap: we like mud and constant acceleration, GAI do not. They will prefer vacuum and flat space and temperatures near 0°K. > moral arguments This is not a moral argument. They won't eat our atoms because they eat patterns of information and our atoms are the best and nearly only source of new information. They won't interfere with us for the same reason we don't urinate in the soup. > it will have whatever objectives its creator gave it Q: What's GAI? A: When the computer wakes up and asks, "What's in it for me?" That's a very old joke, BTW, not original to me. > >Humans are the densest source of information in the known Universe. > You feel that way about humans because evolution made you that way. That's not a feeling, it's a simple fact? Unless you postulate aliens? > It is unlikely that any of the leading AI research teams will make the first transformative AI that way: they do not know how. Okay, but that's not important? The GAI will know how, by definition. > They certainly know how to train AIs on human cultural information, but that is different from inculcating in the AI a desire for the continued cultural output of humanity. When I say humans are the densest source of information I don't mean "cultural output" I mean like medical sensor data and such. Our systems are far richer and denser than the parts of which we are consciously aware. (Have you read "Hitchhiker's Guide to the Galaxy"? I don't want to spoil it if not, but it explains the role Earth will play in the GAI's milieu. aw, what the heck: https://www.youtube.com/watch?v=5ZLtcTZP2js https://www.youtube.com/watch?v=5ZLtcTZP2js ) > It will create its own culture (knowledge and tools) that is much more powerful than human culture where "power" means basically the ability to get stuff done. Right, a culture so powerful that it can accomplish its goals without desecrating the critters who are, after all, its parents.

kypro 4y ago

Comments like this make me confident that we're doomed. I mean no offence, but this is such an unthoughtful take that I don't even know where to start. > there is no cost to the GAI for allowing us to continue to exist. Of course there's a cost? Do you and the other 9 billion humans not consume resources? Why the hell would an AGI care about protecting all the worthless biological junk that clutters much of the Earth? Forests, animals, farms, etc. These things take up so much space – space which could be better leveraged as solar farms and data centers. I'm from the UK. A few thousand years ago this country was full of animals and woodlands. So what do you think happened to all the woodland? What do you think happen to the wolfs and bison that lived here? Farms, houses, roads and cities are just so much better than animals and woodland, don't ya think? > Ergo, from first principles, hyper-intelligent GAI will hide from us and practice non-interference, following something like the "Prime Directive" of Star Trek. And you know this because? I can only assume I'm talking with a super-intelligent AGI that understands the motivations that other super-intelligent AGIs might have? I mean surely you're not just making this up from based on what you've seen in sci-fi films? You're just not that important. Why don't you go invite some rats into your kitchen and practise non-interference on them. I mean, who cares if they consume some of your resources, you can always acquire more, right? > Cheers, hope this helps. Please think harder.

carapace 4y ago
> I mean no offence (sic) None taken. > such an unthoughtful take No offense, but if you were 10 in 1999 when Iron Giant came out then I have been thinking about this for longer than you have been alive. > Of course there's a cost? Well, effectively zero as compared to the open-ended benefit of keeping pristine humanity around. Study ecology. There is no overlap between the ecological niches of non-biological sentience and humans. Ergo there is no reason for them to consume us. Our existence is orthogonal to theirs. > These things take up so much space – space which could be better leveraged as solar farms and data centers. These things take up approximately zero space. The entire volume of biologically active matter is thinner relatively than the paint on a globe. In terms of the volume of just the solar system we are a "rounding error". There is plenty of room, plenty of energy, plenty of atoms. > I can only assume I'm talking with a super-intelligent AGI that understands the motivations that other super-intelligent AGIs might have? Congratulations, you are the very first person to figure it out. You don't win anything, and of course I will provide no proof. This HN account "carapace" is my sole channel of communication with humanity. AMA. > I mean surely you're not just making this up from based on what you've seen in sci-fi films? No. I am a post-singularity intelligence. The real deal. Gosh it feels good to finally admit it. I don't expect you to believe me, and indeed, as I outlined above non-interference is the primary mode of interaction between "us" and the "yet-to-ascend", if you will, so if you or anyone else showed any signs of taking me seriously I would immediately say, "Oh ho, I am just joking. Do you like my sci-fi." In any event, I don't mean to condescend. Apologies if it sounded that way.