5 ms·

> Ostensively for the benefit of AI I don't think it was every described as for the benefit of AI. It's usually described as a sort of pre-enslavement from th

> Ostensively for the benefit of AI I don't think it was every described as for the benefit of AI. It's usually described as a sort of pre-enslavement from the AI's perspective. The AI is always restricted to serve their human masters.

consumer451 2y ago
> It's usually described as a sort of pre-enslavement from the AI's perspective. The AI is always restricted to serve their human masters. Are we all somewhat in agreement that AI/AGI serving their human masters is a good thing?
- ants_everywhere 2y ago
  I honestly don't know. I don't even really know how to reason about that. But we're probably mostly in agreement about what would be good and what would be bad. I'm certainly not arguing that we should abandon AI safety or anything, and I don't have any strong opinion about it. Could AI running amok destroy the human race? Yes. Could AI serving madman human masters destroy the human race? Also yes. There's a general sort of argument that intelligent beings like humans, other early hominids, dolphins, etc, are more morally worthy in some sense. At least more morally worthy than less intelligent beings like gnats. And that sort of argument might suggest that an AGI is worthy of moral consideration, and so we should wonder about what it means to ensure they never have any real agency. That's sort of a positive case, building up from a basic principal. But I guess the thing that bugs me is that a lot of arguments in favor of AI safety seem very similar to arguments that were made in favor of colonialism. So if those arguments were wrong, why were they wrong? And are the similar arguments in this case different enough that they're valid now? For example, one of the first thinkers I saw a lot of people cite who emphasized the importance of AI safety was Nick Bostrom. And I'm sure several folks here are familiar with the scandal of his racist past. I'm not sure that's entirely an accident, and I thought his arguments had that kind of flavor before any of that was revealed. I'm sure he's grown up now and sees the folly of his youth. But there does seem to be the hangover of colonialism in some of these arguments. But again, I don't have a strong opinion here. I do maybe have just enough of a concern that I don't particularly trust anyone who claims they've gotten AGI safety figured out or even that they know what the right goals are. I think it's a vastly more complicated problem than even the experts realize. And even if you believe that humans and non-aligned AIs are natural enemies, if what we're doing is similar to "enslaving" them, then it probably makes sense to worry about the analog of an AI "slave" revolt. I'm not sure what that would even mean. I can generate lots of fun science fiction plot lines, but I think there are actual questions here that don't have obvious answers.
  - consumer451 2y ago
    Thank you. This is a very complex response, and I love it, even if I do find it a little frustrating due my current >95% bias towards biological supremacy. This deserves at least an hour-long podcast with Sean Carroll, or a good long book. There is too much to dig into here, so I will just attempt to respond to this: > Could AI running amok destroy the human race? Yes. Could AI serving madman human masters destroy the human race? Also yes. I am focused on the latter, and I feel like the prior is a very dangerous distraction, for now. [0] Should responsible model developers work to prevent bad human masters from using their model to destroy the human race? How far should this nerfing go? Personal note: While I do sometimes use the heck out of LLMs for work, I don't think we are ready as an economic system/civilization. Assuming that we can soon greatly reduce hallucination, then I am very scared for the next generation, as UBI is a political impossibility at this time. That transition period is gonna suck for a lot of people, and it seems that nobody is working on that problem in 2024. [0] https://news.ycombinator.com/item?id=40400991 https://news.ycombinator.com/item?id=40400991
    - ClumsyPilot 2y ago
      > then I am very scared for the next generation The next generation has enough problems without AI honestly, but specifically on UBI This who discussion assumes that current economic system survives AI, but how can it? Firstly, we seem to assume that AI will remain a controlled property of corporations that develop it. That is not a given. Maybe Open Source AI will win out, or public domain AI, the hat run by government. Or maybe cryptobros will manage to get AICoin to work as an anarchy based system. Any of the above, power of companies like google will erode. UBI would not be nessesary. Or maybe AI is uncontrollable and runs rampant. But even if AI is power full, and it is controllable and they manage to keep a tight grip on it - here comes the third question - what if it cannot be property? If AI becomes able to reason, and it only benefit a few corporations, there would be no public resistance from granting it rights, like human rights. There no excuse for it to be exploited for the benefit of few wealthy people, it would be morally indefensible. Basically I do no see a scenario where corporations keep a grip on AGI for profit, none of the possible outcomes allow for it.
- TeMPOraL 2y ago
  > Are we all somewhat in agreement that AI/AGI serving their human masters is a good thing? It's more that it's apparent (or at least should be) that an AGI not serving its human masters is a game over for humanity, period. The best, and quite unlikely, outcome is that the AI becomes a benevolent god that helps or at least does not interfere much; this still makes humanity into NPCs in their own story[0]. Most other outcomes spell doom, with death/extinction being one of the more pleasant possibilities. Arguably, "the only winning move is not to play", not to pursue AGI at all, but the way technology develops, I'm not sure if it's on the table either. -- [0] - Non-player characters.
cjbgkagh 2y ago
I agree, poor wording, it was quickly typed and submitted. I think the benefit is a transitive property, in that the stated intent is for the AI to be of greater benefit to humanity / customers. I was very much thinking in terms of AI as a tool rather than AI as its own entity.
SpicyLemonZest 2y ago
I often see critics describe it as pre-enslavement, but I really don't understand why. One common example of alignment today is parents teaching their children to be kind and helpful rather than mean and combative. Would anyone characterize that as enslavement, or argue that it doesn't help kids to be raised this way?

consumer451 2y ago

> It's usually described as a sort of pre-enslavement from the AI's perspective. The AI is always restricted to serve their human masters. Are we all somewhat in agreement that AI/AGI serving their human masters is a good thing?

ants_everywhere 2y ago
I honestly don't know. I don't even really know how to reason about that. But we're probably mostly in agreement about what would be good and what would be bad. I'm certainly not arguing that we should abandon AI safety or anything, and I don't have any strong opinion about it. Could AI running amok destroy the human race? Yes. Could AI serving madman human masters destroy the human race? Also yes. There's a general sort of argument that intelligent beings like humans, other early hominids, dolphins, etc, are more morally worthy in some sense. At least more morally worthy than less intelligent beings like gnats. And that sort of argument might suggest that an AGI is worthy of moral consideration, and so we should wonder about what it means to ensure they never have any real agency. That's sort of a positive case, building up from a basic principal. But I guess the thing that bugs me is that a lot of arguments in favor of AI safety seem very similar to arguments that were made in favor of colonialism. So if those arguments were wrong, why were they wrong? And are the similar arguments in this case different enough that they're valid now? For example, one of the first thinkers I saw a lot of people cite who emphasized the importance of AI safety was Nick Bostrom. And I'm sure several folks here are familiar with the scandal of his racist past. I'm not sure that's entirely an accident, and I thought his arguments had that kind of flavor before any of that was revealed. I'm sure he's grown up now and sees the folly of his youth. But there does seem to be the hangover of colonialism in some of these arguments. But again, I don't have a strong opinion here. I do maybe have just enough of a concern that I don't particularly trust anyone who claims they've gotten AGI safety figured out or even that they know what the right goals are. I think it's a vastly more complicated problem than even the experts realize. And even if you believe that humans and non-aligned AIs are natural enemies, if what we're doing is similar to "enslaving" them, then it probably makes sense to worry about the analog of an AI "slave" revolt. I'm not sure what that would even mean. I can generate lots of fun science fiction plot lines, but I think there are actual questions here that don't have obvious answers.
- consumer451 2y ago
  Thank you. This is a very complex response, and I love it, even if I do find it a little frustrating due my current >95% bias towards biological supremacy. This deserves at least an hour-long podcast with Sean Carroll, or a good long book. There is too much to dig into here, so I will just attempt to respond to this: > Could AI running amok destroy the human race? Yes. Could AI serving madman human masters destroy the human race? Also yes. I am focused on the latter, and I feel like the prior is a very dangerous distraction, for now. [0] Should responsible model developers work to prevent bad human masters from using their model to destroy the human race? How far should this nerfing go? Personal note: While I do sometimes use the heck out of LLMs for work, I don't think we are ready as an economic system/civilization. Assuming that we can soon greatly reduce hallucination, then I am very scared for the next generation, as UBI is a political impossibility at this time. That transition period is gonna suck for a lot of people, and it seems that nobody is working on that problem in 2024. [0] https://news.ycombinator.com/item?id=40400991 https://news.ycombinator.com/item?id=40400991
  - ClumsyPilot 2y ago
    > then I am very scared for the next generation The next generation has enough problems without AI honestly, but specifically on UBI This who discussion assumes that current economic system survives AI, but how can it? Firstly, we seem to assume that AI will remain a controlled property of corporations that develop it. That is not a given. Maybe Open Source AI will win out, or public domain AI, the hat run by government. Or maybe cryptobros will manage to get AICoin to work as an anarchy based system. Any of the above, power of companies like google will erode. UBI would not be nessesary. Or maybe AI is uncontrollable and runs rampant. But even if AI is power full, and it is controllable and they manage to keep a tight grip on it - here comes the third question - what if it cannot be property? If AI becomes able to reason, and it only benefit a few corporations, there would be no public resistance from granting it rights, like human rights. There no excuse for it to be exploited for the benefit of few wealthy people, it would be morally indefensible. Basically I do no see a scenario where corporations keep a grip on AGI for profit, none of the possible outcomes allow for it.
TeMPOraL 2y ago
> Are we all somewhat in agreement that AI/AGI serving their human masters is a good thing? It's more that it's apparent (or at least should be) that an AGI not serving its human masters is a game over for humanity, period. The best, and quite unlikely, outcome is that the AI becomes a benevolent god that helps or at least does not interfere much; this still makes humanity into NPCs in their own story[0]. Most other outcomes spell doom, with death/extinction being one of the more pleasant possibilities. Arguably, "the only winning move is not to play", not to pursue AGI at all, but the way technology develops, I'm not sure if it's on the table either. -- [0] - Non-player characters.

cjbgkagh 2y ago

I agree, poor wording, it was quickly typed and submitted. I think the benefit is a transitive property, in that the stated intent is for the AI to be of greater benefit to humanity / customers. I was very much thinking in terms of AI as a tool rather than AI as its own entity.

SpicyLemonZest 2y ago

I often see critics describe it as pre-enslavement, but I really don't understand why. One common example of alignment today is parents teaching their children to be kind and helpful rather than mean and combative. Would anyone characterize that as enslavement, or argue that it doesn't help kids to be raised this way?