4 ms·

I am really not sure if I buy this. OpenAI has ignited a space race between Google and Microsoft. It seems to me their whole reason for existing was to break th

I am really not sure if I buy this. OpenAI has ignited a space race between Google and Microsoft. It seems to me their whole reason for existing was to break the GUI out of Bell Labs, to draw an analogy. If they think AGI is possible and an existential risk, surely they must realize they can’t hold back the ocean. 1000 companies will race to the finish line if one drops out of the race. On the other hand, I believe the Fermi Paradox suggests that AGI is impossible. Machine intelligence would be the perfect vector for rapid galactic conquest. The fact that our planet isn’t covered in solar panels and compute paperclips suggests that super intelligence is either once in a galaxy rare or impossible.

GeorgeTirebiter 4y ago
What about the scenario that Humans are the First ? Once-in-a-galaxy rare (more like once in a visible universe rare; see: https://arxiv.org/pdf/1806.02404.pdf https://arxiv.org/pdf/1806.02404.pdf ) The more I understand how we got life from chemicals (it truly is nothing short of miraculous, even if you like I do not believe in supernatural beginnings), the more I come away with how unlikely it is that we are here. But, here we are! And, so, WE get to populate the galaxy and maybe other galaxies with our Humanity. Lucky us. There are plenty of other scenarios (they are hiding; they are waiting until we develop Warp Drive, whatever), but applying Occam's Razor says: WE are the first intelligent species in this part of the Universe. And, as a consequence, WE have a chance of using all these atoms out there for our purposes. To me, this awesome responsibility trumps Little Green Men and their supposed machines populating the galaxies.
- dougmwne 4y ago
  We have no population of life supporting worlds to compare to, but we could take the timeline of events on our own planet as a way to estimate their probability. Each year something doesn’t happen makes it all the more rare of an event. As far as we can tell, DNA and single celled life evolved almost as soon as the planet cooled down enough for liquid water. There may have been a short gap of a lifeless water world, but it does appear to be quite short. Complex multicellular life seems to be extremely rare considering that the Cambrian explosion took billions of years to kick off. Once it got going it moved very fast filling our world with an incredible diversity of life. Intelligence seems somewhat common considering that it exists in birds, mammals and octopi, something that is probably inevitable once you have multicellular organisms. Human level intelligence might be a bit rare, but probably not particularly. And once you have humans, the race to the stars seems inevitable. So of all of this, the Cambrian seems mostly likely to be the great filter unless intelligence naturally causes its own extinction in short order.
reducesuffering 4y ago
Unless the Fermi Paradox is rapid societal destruction from AGI? This is what Eliezer believes who was just yesterday hanging out with Sam Altman. Civilizations can't get out of their solar system because they invent unaligned AGI first that paperclip-optimizes them out of existence for something else.
- dougmwne 4y ago
  But if that happens then why would the AGI stop at one planet? There’s a whole universe to optimize!

GeorgeTirebiter 4y ago

What about the scenario that Humans are the First ? Once-in-a-galaxy rare (more like once in a visible universe rare; see: https://arxiv.org/pdf/1806.02404.pdf https://arxiv.org/pdf/1806.02404.pdf ) The more I understand how we got life from chemicals (it truly is nothing short of miraculous, even if you like I do not believe in supernatural beginnings), the more I come away with how unlikely it is that we are here. But, here we are! And, so, WE get to populate the galaxy and maybe other galaxies with our Humanity. Lucky us. There are plenty of other scenarios (they are hiding; they are waiting until we develop Warp Drive, whatever), but applying Occam's Razor says: WE are the first intelligent species in this part of the Universe. And, as a consequence, WE have a chance of using all these atoms out there for our purposes. To me, this awesome responsibility trumps Little Green Men and their supposed machines populating the galaxies.

dougmwne 4y ago
We have no population of life supporting worlds to compare to, but we could take the timeline of events on our own planet as a way to estimate their probability. Each year something doesn’t happen makes it all the more rare of an event. As far as we can tell, DNA and single celled life evolved almost as soon as the planet cooled down enough for liquid water. There may have been a short gap of a lifeless water world, but it does appear to be quite short. Complex multicellular life seems to be extremely rare considering that the Cambrian explosion took billions of years to kick off. Once it got going it moved very fast filling our world with an incredible diversity of life. Intelligence seems somewhat common considering that it exists in birds, mammals and octopi, something that is probably inevitable once you have multicellular organisms. Human level intelligence might be a bit rare, but probably not particularly. And once you have humans, the race to the stars seems inevitable. So of all of this, the Cambrian seems mostly likely to be the great filter unless intelligence naturally causes its own extinction in short order.

reducesuffering 4y ago

Unless the Fermi Paradox is rapid societal destruction from AGI? This is what Eliezer believes who was just yesterday hanging out with Sam Altman. Civilizations can't get out of their solar system because they invent unaligned AGI first that paperclip-optimizes them out of existence for something else.

dougmwne 4y ago
But if that happens then why would the AGI stop at one planet? There’s a whole universe to optimize!