4 ms·

> Those aren't actually limitations preventing us from becoming godlike. Those are limitations on us becoming more populous and collectively more experienced, w

> Those aren't actually limitations preventing us from becoming godlike. Those are limitations on us becoming more populous and collectively more experienced, which I argue isn't the same thing at all. If you could grow your own brain as large as you wanted, and live as long as you wanted with the physical robustness and energy of a 25-year-old, that alone seems like a road to godhood, though perhaps a slow one. If you could download knowledge directly, as another commenter suggests, that becomes a much faster road. > Apotheosis in the context of AI safety as well as for humankind is really about the accumulation of power, rather than the accumulation of knowledge. The extent of our apotheosis is limited only by what we can control, not by what we know. If one country, even a smallish one, has 50-100 additional years of technological advancement, I think they might be able to use this knowledge to gain a lot more power. I'm pretty sure history has borne this out. Do you think we're at the limit of what new technology can do for military strength? Materials science (armor, weapons), biotechnology, energy and battery technology, cybersecurity, psychology and propaganda, lasers and anti-missile defense systems... Suppose a country like North Korea had 100 Einsteins working on such things for a few years under the radar. I suppose it might be difficult for them to hide the construction of their army of thousands of robots invincible to all but the most powerful bombs... but then again, maybe they could do it underground, directly underneath some large innocuous manufacturing operation. And, of course, the cyber- and bio-weapons efforts would be very easy to conceal.

psychoslave 7y ago
>If you could grow your own brain as large as you wanted, and live as long as you wanted with the physical robustness and energy of a 25-year-old, that alone seems like a road to godhood, though perhaps a slow one. If you could download knowledge directly, as another commenter suggests, that becomes a much faster road. A larger brain don't necessarily provide a better survival advantage. It's well known that average Homo Neanderthalensis had larger brains than our modern average follow humans. And if you don't survive, you are nowhere toward the road to thaumaturgic existence. What is the trade off is the most important question, as anywhere.
TheOtherHobbes 7y ago
You can't grow your brain as large as you want. There are physical limits on power consumption, power dissipation, and organisational complexity. And size doesn't even correlate with intelligence. It's even debatable if you can grow an Einstein. What if GI is NP-hard? What if the belief that GI - never mind consciousness - can be modelled with our version of computing hardware is just plain wrong? What if a GI System is uncopyable for practical or philosophical reasons? The singularity looks more like plain old religious thinking than anything that could be recognised as real science. "What if we made a golem? We could do anything! What if we put ourselves into the golem? We could live forever!" And so on.
- edejong 7y ago
  Thank you so much. This comment is golden. I’ve been pushing similar thoughts for years. Also, we should take into consideration the power of our communication structure which has evolved over hundreds of generations. Even if there would be true generalized Artificial intelligence, it is not unreasonable to think that the formation of the right thought framework for those systems to function effectively will require a substantial amount of time. It is also reasonable to think that such change would probably be in symbiosis with human intelligence. This change would not be sudden, but happen over many generations. It is my opinion that this change has already started at least a generation ago.
  - pdimitar 7y ago
    > Even if there would be true generalized Artificial intelligence, it is not unreasonable to think that the formation of the right thought framework for those systems to function effectively will require a substantial amount of time. Who is disputing that? I am 99.9% certain that this is true. The point of the singularity isn't that AI is coming next summer -- it's that from one developmental point and on the intelligence and abilities explode exponentially. To get to that stage though, thousands of years might pass. > It is also reasonable to think that such change would probably be in symbiosis with human intelligence. Now that is not likely. At all. They have to have the same motivation, interests, emotions and needs like us. The chances of an artificial race being developed to cover all these at once are practically zero. A race not motivated by the same things we are is practically impossible to sympathise with our own motives.

psychoslave 7y ago

>If you could grow your own brain as large as you wanted, and live as long as you wanted with the physical robustness and energy of a 25-year-old, that alone seems like a road to godhood, though perhaps a slow one. If you could download knowledge directly, as another commenter suggests, that becomes a much faster road. A larger brain don't necessarily provide a better survival advantage. It's well known that average Homo Neanderthalensis had larger brains than our modern average follow humans. And if you don't survive, you are nowhere toward the road to thaumaturgic existence. What is the trade off is the most important question, as anywhere.

TheOtherHobbes 7y ago

You can't grow your brain as large as you want. There are physical limits on power consumption, power dissipation, and organisational complexity. And size doesn't even correlate with intelligence. It's even debatable if you can grow an Einstein. What if GI is NP-hard? What if the belief that GI - never mind consciousness - can be modelled with our version of computing hardware is just plain wrong? What if a GI System is uncopyable for practical or philosophical reasons? The singularity looks more like plain old religious thinking than anything that could be recognised as real science. "What if we made a golem? We could do anything! What if we put ourselves into the golem? We could live forever!" And so on.

edejong 7y ago
Thank you so much. This comment is golden. I’ve been pushing similar thoughts for years. Also, we should take into consideration the power of our communication structure which has evolved over hundreds of generations. Even if there would be true generalized Artificial intelligence, it is not unreasonable to think that the formation of the right thought framework for those systems to function effectively will require a substantial amount of time. It is also reasonable to think that such change would probably be in symbiosis with human intelligence. This change would not be sudden, but happen over many generations. It is my opinion that this change has already started at least a generation ago.
- pdimitar 7y ago
  > Even if there would be true generalized Artificial intelligence, it is not unreasonable to think that the formation of the right thought framework for those systems to function effectively will require a substantial amount of time. Who is disputing that? I am 99.9% certain that this is true. The point of the singularity isn't that AI is coming next summer -- it's that from one developmental point and on the intelligence and abilities explode exponentially. To get to that stage though, thousands of years might pass. > It is also reasonable to think that such change would probably be in symbiosis with human intelligence. Now that is not likely. At all. They have to have the same motivation, interests, emotions and needs like us. The chances of an artificial race being developed to cover all these at once are practically zero. A race not motivated by the same things we are is practically impossible to sympathise with our own motives.