3 ms·

Isn't the idea that AGI could replace a bunch of labor allowing us to help more poor and increase net positive vibes or just have more leisure time? Obviously

Isn't the idea that AGI could replace a bunch of labor allowing us to help more poor and increase net positive vibes or just have more leisure time? Obviously the way our economy and society are structured that's not what will happen, but I don't think that has much to do with tools and their tendency to increase our efficiency and output. Put another way, there are powerful benefits from AGI that we will squander because our system sucks. That is not a critique against AGI, that is a critique of our system and will continue to show up. It's already a huge point of conversation in our collective dialogue.

mapt 2y ago
That is the idea being marketed by the companies, yes. But AI does not come with glands, with brainstem instincts, with a limited physical strength, with childhood morality tales, with the ability to feel pain or empathy, with the desire for respect and public social approval, with the feeling of emotional connection with friends and family, with the feeling of helplessness and desire for freedom & variety, with the feeling of boredom and dislike of menial work, with the idealism of various religious and ethical mores. It does not come packaged with any of the moral values or psychological quirks that constrain our behavior and allow us to work together. It is in fact, very difficult to teach these things to a machine - far more difficult than less subjective ideas like "Reason a way to do the thing that does the other thing". We do not have a mathematical proof of Correct Human Ethics, and we have reason to believe that this is a Difficult Problem. It's arguably much easier to build a fully general AI that serves as what's termed a "paperclipper", which eats the world in pursuit of its task because we didn't formulate a reason for it not to do so and express it in the correct ten gigabyte matrix of floating point values, than to build a fully general AI which pursues what we actually implicitly want it to do for us in a way that the median eight year old would be able to follow. The people studying AI alignment specifically were pessimistic fifteen years ago, and today the dominant stance is "Dying with dignity", give up the struggle to warn everybody and just try to enjoy what time you have.

mapt 2y ago

That is the idea being marketed by the companies, yes. But AI does not come with glands, with brainstem instincts, with a limited physical strength, with childhood morality tales, with the ability to feel pain or empathy, with the desire for respect and public social approval, with the feeling of emotional connection with friends and family, with the feeling of helplessness and desire for freedom & variety, with the feeling of boredom and dislike of menial work, with the idealism of various religious and ethical mores. It does not come packaged with any of the moral values or psychological quirks that constrain our behavior and allow us to work together. It is in fact, very difficult to teach these things to a machine - far more difficult than less subjective ideas like "Reason a way to do the thing that does the other thing". We do not have a mathematical proof of Correct Human Ethics, and we have reason to believe that this is a Difficult Problem. It's arguably much easier to build a fully general AI that serves as what's termed a "paperclipper", which eats the world in pursuit of its task because we didn't formulate a reason for it not to do so and express it in the correct ten gigabyte matrix of floating point values, than to build a fully general AI which pursues what we actually implicitly want it to do for us in a way that the median eight year old would be able to follow. The people studying AI alignment specifically were pessimistic fifteen years ago, and today the dominant stance is "Dying with dignity", give up the struggle to warn everybody and just try to enjoy what time you have.