10 ms·

I think what a lot of people here are missing is that the priorities of E.A. are fluid by design. Yes, existential risk of general AI might be small; but even

I think what a lot of people here are missing is that the priorities of E.A. are fluid by design. Yes, existential risk of general AI might be small; but even fewer people are working on it: only about one hundred, worldwide. Similarly, the reason E.A. stepped off the climate change train is because it was a popular issue, which means the marginal benefit of one person contributing decreases. When more people direct attention to AI safety, another area will be the one where E.A. can contribute most, and the focus will shift.

jefftk 4y ago
> Yes, existential risk of general AI might be small; but even fewer people are working on it: only about one hundred, worldwide. You can make that argument, but I think the median view among people working on AI alignment is that the risk is quite high, in the range of >20% chance of disaster within their lifetime.
gnramires 4y ago
Indeed not only fluid, but also open to criticism. I've been always quite unsure of how to quantify AI risk. My conclusion at the moment is that it's a complex risk. I don't think singularity is plausible in any way or FOOM. Still, the changes to society are absolutely hard to grasp. Humans may face (economical) obsolescence in a few decades[1] (humans need not apply[2]). That puts us in a delicate situation: if humans are economically obsolete, a society that only values economic value (and not consciousness per se) will have all reasons to slowly but surely get rid of humans (and animals). We need a revolution of meaning and ethical understanding: the understanding of the fundamental value of consciousness, and to structure our lives and all institutions around it (each individual needs to have a good understanding of such ethics and meaning so that society as a whole is robust). I think formalization of ethics (which I'm a proponent of) and progress in metaphysics is going to contribute as well to this end. I think EA is simply part of this new understanding of ourselves and the Universe, a sort of coming out of a dark age -- and there's the opportunity to create a lasting society amazing for everyone. [1] My personal guess-estimate is about 50(-20/+40) years to complete obsolescence, although various researchers have wildly different estimates. [2] https://www.youtube.com/watch?v=7Pq-S557XQU https://www.youtube.com/watch?v=7Pq-S557XQU

jefftk 4y ago

> Yes, existential risk of general AI might be small; but even fewer people are working on it: only about one hundred, worldwide. You can make that argument, but I think the median view among people working on AI alignment is that the risk is quite high, in the range of >20% chance of disaster within their lifetime.

gnramires 4y ago

Indeed not only fluid, but also open to criticism. I've been always quite unsure of how to quantify AI risk. My conclusion at the moment is that it's a complex risk. I don't think singularity is plausible in any way or FOOM. Still, the changes to society are absolutely hard to grasp. Humans may face (economical) obsolescence in a few decades[1] (humans need not apply[2]). That puts us in a delicate situation: if humans are economically obsolete, a society that only values economic value (and not consciousness per se) will have all reasons to slowly but surely get rid of humans (and animals). We need a revolution of meaning and ethical understanding: the understanding of the fundamental value of consciousness, and to structure our lives and all institutions around it (each individual needs to have a good understanding of such ethics and meaning so that society as a whole is robust). I think formalization of ethics (which I'm a proponent of) and progress in metaphysics is going to contribute as well to this end. I think EA is simply part of this new understanding of ourselves and the Universe, a sort of coming out of a dark age -- and there's the opportunity to create a lasting society amazing for everyone. [1] My personal guess-estimate is about 50(-20/+40) years to complete obsolescence, although various researchers have wildly different estimates. [2] https://www.youtube.com/watch?v=7Pq-S557XQU https://www.youtube.com/watch?v=7Pq-S557XQU