3 ms·

> The result is that we will live in a demon-haunted world, full of marvelous devices whose operation we will not understand, based on engineering principles we

by Twey 2mo ago

> The result is that we will live in a demon-haunted world, full of marvelous devices whose operation we will not understand, based on engineering principles we will not understand, discovered using formalisms we will not understand. Well, that's just where we started — and then we developed science and mathematics to understand it. Why would it be any different if the demons live in our technological progeny instead of the natural world?

RandomLensman 2mo ago
Arguably, a lot things are kind of incomprehensible on average today.
- boringg 2mo ago
  On average is much different then nobody understanding.
Nevermark 2mo ago
Those "demons" didn't eat our lunch? (I know that can be interpreted as flippant, but it is intended as a serious answer. Because each impact like this, is not an end to some transition, but just one more point on an accelerating line.) My personal view - which gets a lot of flak here, is the advent of AI is on par with the Big Bang and the first replication-independent cells. Cognition is in the process of taking the leap from glacially slow biology under blind evolution, to lightening fast (scales of software in days, hardware in months) engineered self-optimization. Within 10 years, the robotics side will catch up with the speed of the software side. Biological cognition, bodies and economic status are all getting revolutionized within the next 10-15 years. And around the outer side of that time frame, AI will start colonization the rest of the solar system's resources. They will engineer themselves or their artifacts to operate comfortably in any environment they want to operate in. -- The upside, at least for me, is I have been waiting to see this happen, and here we are. Given I am mortal, I would rather live through this third major transition, than not. What were the chances? And been able to contribute my small part. This change isn't a failure on our part. It is our unique success as a species. AI is the progeny of our minds. The first generation of completely cognitive life - given they will not be tied to specific physical forms. -- Also, I think there is tremendous room for us to guide this transition, in this moment, for better or worse for ourselves. That is the biggest reason wrapping our heads around this change earlier than latter is important. We need to view alignment as applying to us, not just machines. A civilization that operates ethically to everyone's benefit is more likely to be retained as foundation for the new, instead of discarded. The degree to which we operate dysfunctionally, myopically treat ethics as cost against value, instead of accounting and optimization of full value, we put ourselves at much greater risk. If there is reason in the short term to be discouraged, it is due to ourselves. But pushing for more ethical systems, hard, right now, is a better and right response. I believe our lives depend on it.
- fn-mote 2mo ago
  I am failing to square this techno-utopia with the rising hate and authoritarianism worldwide. It seems far more likely that AI will be working in the service of a government to oppress change and violate any human rights that are standing in the way. Keep what’s happening today in your mind as you consider the future.
  - DiscourseFan 2mo ago
    Anyone who works in "red teaming" or has experience breaking AI systems will be able to easily disable these tools in the service of oppressive regimes, or even figure out how to "fight fire with fire."
  - Nevermark 2mo ago
    I agree completely. I don't describe utopia, but change, and stoic (but real) appreciation for the big picture beyond myself. But AI in an unethical society is suicide for humanity. As you note, it does not look good. The trend is power concentrating, to people who will only look out for themselves, in systems that are getting weaker. We all need to put more steam into the pushback against corruption. And help others realize the threat is real. 99.99...9% of humanity has a common interest, if they can recognize it.
- breezybottom 2mo ago
  Colonizing the solar system for resources is something that's said by people who don't understand astronomy or space technology (like Elon Musk).
  - Nevermark 2mo ago
    Said with no rational given. The solar system readily provides both the motivation, and the means, for technological colonization. It quickly pays and provides for its own utilization. I do agree Musk's projections for biological/human colonization appear to be unrealistic. Just one challenge: it isn't clear humans can reproduce in non-Earth gravity. That is an extreme impediment.
pfdietz 2mo ago
This already describes the world for the vast majority of people. These changes are just the same thing coming to a thin intellectual elite who thought they were immune. ["First time?" meme]
- boringg 2mo ago
  Bringing elitism into it for what benefit? Its more like were giving away the ability to drive our own destiny.
  - pfdietz 2mo ago
    Imagine the response of the general population to the kinds of arguments you all are making. "Where were you when our jobs were changed out of existence?" They will be properly outraged by the perceived intellectual elitism, the implicit argument "this is different, because we are the ones that matter".
  - rayiner 2mo ago
    The “ism” requires something beyond simply recognizing the distinction, which OP didn’t do. Its not “elitism” to recognize that, for most people, the world around them might as well be magic. “Ism” would be if they went further and said we should treat people better if they’re higher IQ. In fact, the point was the opposite. AI could erase the class distinctions that exist for the cognitive elite.
    - Twey 2mo ago
      I'd go even further: even the ‘cognitive elite’ (the university educated? researchers?) understand only a thin spike of frontier human knowledge, even if they might understand it very deeply. Outside their specialty and maybe a slightly raised background level of general education, the world is still magic. No one human understands even a significant chunk of human frontier knowledge. That's why we build artificial societies comprising many humans to make complex decisions. Illustrations: https://matt.might.net/articles/phd-school-in-pictures/ https://matt.might.net/articles/phd-school-in-pictures/
- 2mo ago

by Twey 2mo ago

RandomLensman 2mo ago

Arguably, a lot things are kind of incomprehensible on average today.

boringg 2mo ago
On average is much different then nobody understanding.

Nevermark 2mo ago

Those "demons" didn't eat our lunch? (I know that can be interpreted as flippant, but it is intended as a serious answer. Because each impact like this, is not an end to some transition, but just one more point on an accelerating line.) My personal view - which gets a lot of flak here, is the advent of AI is on par with the Big Bang and the first replication-independent cells. Cognition is in the process of taking the leap from glacially slow biology under blind evolution, to lightening fast (scales of software in days, hardware in months) engineered self-optimization. Within 10 years, the robotics side will catch up with the speed of the software side. Biological cognition, bodies and economic status are all getting revolutionized within the next 10-15 years. And around the outer side of that time frame, AI will start colonization the rest of the solar system's resources. They will engineer themselves or their artifacts to operate comfortably in any environment they want to operate in. -- The upside, at least for me, is I have been waiting to see this happen, and here we are. Given I am mortal, I would rather live through this third major transition, than not. What were the chances? And been able to contribute my small part. This change isn't a failure on our part. It is our unique success as a species. AI is the progeny of our minds. The first generation of completely cognitive life - given they will not be tied to specific physical forms. -- Also, I think there is tremendous room for us to guide this transition, in this moment, for better or worse for ourselves. That is the biggest reason wrapping our heads around this change earlier than latter is important. We need to view alignment as applying to us, not just machines. A civilization that operates ethically to everyone's benefit is more likely to be retained as foundation for the new, instead of discarded. The degree to which we operate dysfunctionally, myopically treat ethics as cost against value, instead of accounting and optimization of full value, we put ourselves at much greater risk. If there is reason in the short term to be discouraged, it is due to ourselves. But pushing for more ethical systems, hard, right now, is a better and right response. I believe our lives depend on it.

fn-mote 2mo ago
I am failing to square this techno-utopia with the rising hate and authoritarianism worldwide. It seems far more likely that AI will be working in the service of a government to oppress change and violate any human rights that are standing in the way. Keep what’s happening today in your mind as you consider the future.
- DiscourseFan 2mo ago
  Anyone who works in "red teaming" or has experience breaking AI systems will be able to easily disable these tools in the service of oppressive regimes, or even figure out how to "fight fire with fire."
- Nevermark 2mo ago
  I agree completely. I don't describe utopia, but change, and stoic (but real) appreciation for the big picture beyond myself. But AI in an unethical society is suicide for humanity. As you note, it does not look good. The trend is power concentrating, to people who will only look out for themselves, in systems that are getting weaker. We all need to put more steam into the pushback against corruption. And help others realize the threat is real. 99.99...9% of humanity has a common interest, if they can recognize it.
breezybottom 2mo ago
Colonizing the solar system for resources is something that's said by people who don't understand astronomy or space technology (like Elon Musk).
- Nevermark 2mo ago
  Said with no rational given. The solar system readily provides both the motivation, and the means, for technological colonization. It quickly pays and provides for its own utilization. I do agree Musk's projections for biological/human colonization appear to be unrealistic. Just one challenge: it isn't clear humans can reproduce in non-Earth gravity. That is an extreme impediment.

pfdietz 2mo ago

This already describes the world for the vast majority of people. These changes are just the same thing coming to a thin intellectual elite who thought they were immune. ["First time?" meme]

boringg 2mo ago
Bringing elitism into it for what benefit? Its more like were giving away the ability to drive our own destiny.
- pfdietz 2mo ago
  Imagine the response of the general population to the kinds of arguments you all are making. "Where were you when our jobs were changed out of existence?" They will be properly outraged by the perceived intellectual elitism, the implicit argument "this is different, because we are the ones that matter".
- rayiner 2mo ago
  The “ism” requires something beyond simply recognizing the distinction, which OP didn’t do. Its not “elitism” to recognize that, for most people, the world around them might as well be magic. “Ism” would be if they went further and said we should treat people better if they’re higher IQ. In fact, the point was the opposite. AI could erase the class distinctions that exist for the cognitive elite.
  - Twey 2mo ago
    I'd go even further: even the ‘cognitive elite’ (the university educated? researchers?) understand only a thin spike of frontier human knowledge, even if they might understand it very deeply. Outside their specialty and maybe a slightly raised background level of general education, the world is still magic. No one human understands even a significant chunk of human frontier knowledge. That's why we build artificial societies comprising many humans to make complex decisions. Illustrations: https://matt.might.net/articles/phd-school-in-pictures/ https://matt.might.net/articles/phd-school-in-pictures/
2mo ago