6 ms·

There are at least three ways I could see an AI being the great filter: 1. An AI bootstraps itself to near-omnipotence and gobbles up all resources in a sphere

There are at least three ways I could see an AI being the great filter: 1. An AI bootstraps itself to near-omnipotence and gobbles up all resources in a sphere around itself which grows at approximately the speed of light (where "all resources" includes "the atoms humans are made out of") 2. Same as (1) but the sphere grows substantially slower than the speed of light 3. Well before you get to the point where any of that is a risk, you get AI massively amplifying human capabilities. When you have millions of people with access to civilization-ending technology, one of them notices a clever chemical reaction which allows you to do isotopic separation without all of that mucking about with centrifuges and publishes it on the internet. Now we have millions of people with nuclear weapons and the ability to cheaply make more of them. This probably does not end with a civilization that survives and expands to the stars. Only in the case of (2) would we actually _see_ a bunch of AIs roving about the universe gobbling up resources.

nathan_compton 4y ago
Pretty sure we'd see them in case 1) and 2).
- JoshuaDavid 4y ago
  In case (1) there would only be a very brief period of time we would be able to see them while also being alive.
  - nathan_compton 4y ago
    Not if they happened to be spreading rapidly but in some relatively distant part of the galaxy.
    - JoshuaDavid 4y ago
      If the sphere started 1 million light years away and is expanding at 0.99c, there's only a 10,000 year period between when you first see the sphere-of-resource-gobbling and when the sphere-of-resource-gobbling reaches you. If it expands considerably slower than c, I agree with you that you can see it coming. For example, (assuming my math is right) if it expands at 0.8c, then the volume of space that has been eaten is equal to the volume of space that can see the sphere-of-resource-gobbling coming, so the "you don't see them coming" is very sensitive to the speed of the front being very close to the speed of light.
      - nathan_compton 4y ago
        "Only a 10,000 year period." As a physicist I find the idea of a uniformly spreading sphere going at 0.99c pretty unlikely, even absurd.
        JoshuaDavid 4y ago
        Which bits do you find absurd? 1. The bit where it's possible to accelerate a bunch of probes from a star system, aimed at other stars, such that they're going 0.99c when they leave 2. The bit where said probes can survive traveling potentially thousands of light years at 0.99c without slowing down or being destroyed by very-high-energy collisions with dust 3. The bit where those probes can decelerate themselves back down when they reach the other star 4. The bit where, once they decelerate back down at the target star, they can then land on a rocky planet in that system, tile the surface of that planet with solar panels, which they then (maybe) use to disassemble the planet into even more solar panels around that star and also more probes (go back to step 1), over a relatively short period of time. 5. The bit where, repeated over a large number of iterations, the above process looks like a sphere of probes, the edge of which is expanding outwards at approximately 0.99c My guess is that your answer will be either (2) or (3), which is legit. Whether the "grabby aliens" explanation holds up is extremely sensitive to the expansion speed though -- either it is possible to expand at very close to the speed of light, or "the great filter is AGI which eats the universe, multiple civilizations which can develop AGI develop in the universe (and we are one such civilization), and we don't see any other expanding civilizations" would be a combination of things that you would not expect to see (implying that that's probably not the great filter).
        nathan_compton 4y ago
        I think it is (among other things) the _sphere_. Imagine being a super AI. You know you might encounter _other_ super AI in the universe, so your priorities aren't to just spread in a uniform _sphere_. You want to target those places with the most resources that you can use most easily. So I expect even if an AI could spread at near the speed of light you'd expect to see them spreading non-uniformly, and therefor visibly. My deeper skepticism about this is the idea of a super AGI. I think we'll create AGI eventually and that they will likely be more capable than people. But I don't think that being more capable translates into being substantially more able to predict the future, manipulate the physical universe, make scientific discoveries, etc. There are plenty of problems that are hard regardless of how smart you are. I also kind of think that the fact that we don't see super AI gobbling up shit in our galaxy or elsewhere suggests strongly that mega-project scale AI just don't happen in this universe for some reason.

nathan_compton 4y ago

Pretty sure we'd see them in case 1) and 2).

JoshuaDavid 4y ago
In case (1) there would only be a very brief period of time we would be able to see them while also being alive.
- nathan_compton 4y ago
  Not if they happened to be spreading rapidly but in some relatively distant part of the galaxy.
  - JoshuaDavid 4y ago
    If the sphere started 1 million light years away and is expanding at 0.99c, there's only a 10,000 year period between when you first see the sphere-of-resource-gobbling and when the sphere-of-resource-gobbling reaches you. If it expands considerably slower than c, I agree with you that you can see it coming. For example, (assuming my math is right) if it expands at 0.8c, then the volume of space that has been eaten is equal to the volume of space that can see the sphere-of-resource-gobbling coming, so the "you don't see them coming" is very sensitive to the speed of the front being very close to the speed of light.
    - nathan_compton 4y ago
      "Only a 10,000 year period." As a physicist I find the idea of a uniformly spreading sphere going at 0.99c pretty unlikely, even absurd.
      - JoshuaDavid 4y ago
        Which bits do you find absurd? 1. The bit where it's possible to accelerate a bunch of probes from a star system, aimed at other stars, such that they're going 0.99c when they leave 2. The bit where said probes can survive traveling potentially thousands of light years at 0.99c without slowing down or being destroyed by very-high-energy collisions with dust 3. The bit where those probes can decelerate themselves back down when they reach the other star 4. The bit where, once they decelerate back down at the target star, they can then land on a rocky planet in that system, tile the surface of that planet with solar panels, which they then (maybe) use to disassemble the planet into even more solar panels around that star and also more probes (go back to step 1), over a relatively short period of time. 5. The bit where, repeated over a large number of iterations, the above process looks like a sphere of probes, the edge of which is expanding outwards at approximately 0.99c My guess is that your answer will be either (2) or (3), which is legit. Whether the "grabby aliens" explanation holds up is extremely sensitive to the expansion speed though -- either it is possible to expand at very close to the speed of light, or "the great filter is AGI which eats the universe, multiple civilizations which can develop AGI develop in the universe (and we are one such civilization), and we don't see any other expanding civilizations" would be a combination of things that you would not expect to see (implying that that's probably not the great filter).
        nathan_compton 4y ago
        I think it is (among other things) the _sphere_. Imagine being a super AI. You know you might encounter _other_ super AI in the universe, so your priorities aren't to just spread in a uniform _sphere_. You want to target those places with the most resources that you can use most easily. So I expect even if an AI could spread at near the speed of light you'd expect to see them spreading non-uniformly, and therefor visibly. My deeper skepticism about this is the idea of a super AGI. I think we'll create AGI eventually and that they will likely be more capable than people. But I don't think that being more capable translates into being substantially more able to predict the future, manipulate the physical universe, make scientific discoveries, etc. There are plenty of problems that are hard regardless of how smart you are. I also kind of think that the fact that we don't see super AI gobbling up shit in our galaxy or elsewhere suggests strongly that mega-project scale AI just don't happen in this universe for some reason.