10 ms·

You're doing the "vaguely gesturing at imagined hypocrisy" thing. You don't have to agree that alignment is a real issue. But for those who do think it's a rea

by mort96 4y ago

You're doing the "vaguely gesturing at imagined hypocrisy" thing. You don't have to agree that alignment is a real issue. But for those who do think it's a real issue, it has nothing to do with morals of individuals or how one should behave interpersonally. People who are worried about alignment issues are worried about the danger unaligned AI poses to humanity; the harm which can be done by some super-intelligent system optimizing for the wrong outcome. There is nothing "ironic" about being worried about that while also being an asshole any more than it's "ironic" for someone concerned about, say, climate change and also be an asshole. People who are afraid of unaligned AI aren't afraid that it will be impolite. I'm tired of people pretending that pointing out imaginary hypocricy is an argument. If you want to complain that someone is being mean, just do that. Don't pretend there's hypocricy involved.

srslack 4y ago
>People who are afraid of unaligned AI aren't afraid that it will be impolite. People who are not afraid of it being impolite are afraid of science fiction stories about intelligence explosions and singularities. That's not a real thing. Not anymore than turning the solar system into paperclips. The "figurehead", if you want to call him that, is saying that everyone is going to die. That we need to ban GPUs. That only "responsible" companies, if that, should have them. We should also airstrike datacenters, apparently. But you're free to disown the MIRI.
- mort96 4y ago
  I don't know why you're telling me this. I'm not trying to convince you that unaligned AI is a problem. That's a separate discussion which I'm not qualified to have.
- myrmidon 4y ago
  Sorry for derailing this a bit, but I would really like to understand your view: You are not concerned about any "rogue AI" scenario, right? What makes you so confident in that? 1) Do you think that AI achieving superhuman cognitive abilities is unlikely/really far away? 2) Do you believe that cognitive superiority is not a threat in general, or specifically when not embodied? 3) Do you think we can trivially and indefinitely keep AI systems under control/"aligned"? Because I view myself ABSOLUTELY not as some kind of AI luddite, but I honestly believe that this is one of the very few credible extinction threats that we face, and I'm counting NEITHER climate change nor nuclear war in that category, for reference.
  - leroy-is-here 4y ago
    A “superhuman” AI is just a machine, a very expensive one. It can be turned off and we control the outputs it has. Why would an AI have the ability to launch nuclear weapons unless we gave it a button? A “superhuman” intelligence is without a body, so we control any interfaces it has access to. The Internet could be accessed, but any attempt to “hack” through the Internet is met by routine packet defenses. The AI is still governed by physical laws and would only have so much “free” computation power to do things like script a hack. Perhaps it could do that kind of thing more efficiently. Maybe in the far, far future when we have androids which can house an AI we will have to worry. But designing a body is one problem. Designing an intelligence is another.
    - iainmerrick 4y ago
      Who's "we"?
      - leroy-is-here 4y ago
        Humanity.
        iainmerrick 4y ago
        Good thing we're all aligned the same way, then!
        leroy-is-here 4y ago
        Alignment doesn’t matter. Humanity is attempting to build artificial intelligence and so it will be done.
    - pixl97 4y ago
      Supercomputers used to be giant machines we had in giant warehouses... Now the phone in your pocket has the computing power of the 1980s walking around with you. Assuming your super intelligence will always be huge is... well not a great assumption. Also superintelligence doesn't need a body itself. It just needs yours. Putin for example has commanded hundreds of thousands of dumbasses to go get themselves killed in Ukraine. In this case does it matter if Putin is flesh and blood, or a processor that lists out commands for others to follow as long as they are willing to listen?
- concordDance 4y ago
  > That's not a real thing. Not currently, no. And LLMs don't seem to be a way of getting there. However, if we do figure out how to get there? We will definitely need to be sure it/they shares our human values. Because we'll have lost control of our destiny just like orangutans have.
- George83728 4y ago
  > intelligence explosions [...] That's not a real thing. Isn't it too early to say? These systems haven't had much time to iterate on themselves yet. The beginning of an exponential curve can look very tame. That said, I'm not terribly concerned about it. After all, the infrastructure is fragile, literally just unplug the computer if it scares you. If things started to get really spooky, banning GPUs and air-striking fabs would be a real option. I think we can probably control this technology, and kill it if we can't. No cause for alarm, at least yet.
  - NumberWangMan 4y ago
    My worry is that as we start wiring non-super-intelligent AI more into our society, we'll make ourselves more and more vulnerable in the case where an AGI actually gets out of control. Pulling the plug on AI may mean causing a lot of chaos or disruption. And who is to tell if we will know when things are out of control, before it's too late? What if the people in charge just get fake reports of everything being fine? What if something smells a bit fishy, but it's always on the side of not quite worrying enough? What if the AIs are really convincing, or make deals with people, or exploit corruption? Not just that, but it may be like fossil fuel dependency -- the more people's livelihoods are intertwined with, and depend on AI, the harder it is to pull the plug. If we need to stop (and I believe we should) it may be easier to do that now before that happens. To just focus on getting use out of the generative AI and narrow AI we already created in areas like medicine, to deal with the massive societal changes it'll bring, and to work on fixing real social problems, which I think are mostly things tech can't solve.
- axus 4y ago
  The sea level being 3 feet higher than it is now isn't a thing either, but we can still imagine ways it could occur and work to prevent them.
- pixl97 4y ago
  > science fiction stories about intelligence explosions Not sure what you mean here... There was already an intelligence explosion. That creature has since captured what we consider full dominion of the earth so much so we named our current age after them.
egeozcan 4y ago
Correct me if I'm wrong, you are replying to the part where the gp says > It's pretty ironic, for people who are supposedly so concerned about "alignment" and "morality" to threaten others right? If not, please ignore my comment. > People who are worried about alignment issues are worried about the danger unaligned AI poses to humanity If you do not see any evidence regarding such threats, though... > There is nothing "ironic" about being worried about that while also being an asshole You may actually see the assholes being a bigger threat to humanity (or human progress, or both), so at least this, IMHO, adds a dimension of irony.
femiagbabiaka 4y ago
The writers of the stochastic parrots paper do not seem to be concerned about unaligned “AI” (text generators) posing a threat to humanity in the way that the doomsayers are. It’s in the title: “stochastic parrot” is a rebuke of those calling LLMs AGI.
- alextheparrot 4y ago
  Somehow once I attended to your tokens “stochastic parrots” the rest of your words became much more predictable
- srslack 4y ago
  Yes, that's correct. It is a slight rebuke, a bit tongue in cheek, but that paper is also old. I grouped them because the groups are, from my perspective, hard to separate. But there are arguably two camps, "AI safety" and "alignment." Perhaps they should work on that, too, form some camps and argue their cases with something defined instead of masquerading under alignment and AI safety depending on the day. But I could also be totally wrong. Until then, I don't believe either are really operating in reality.
  - femiagbabiaka 4y ago
    IMO the conflation of the two is a purposeful move on the part of the AGI doomsday crowd, since they completely lack scientific rigor otherwise, they cite in bad faith. Timnit Gebru talks about it at length here: https://youtu.be/jAHRbFetqII https://youtu.be/jAHRbFetqII.
tivert 4y ago
> People who are worried about alignment issues are worried about the danger unaligned AI poses to humanity; the harm which can be done by some super-intelligent system optimizing for the wrong outcome. But isn't "alignment" in these cases more about providing answers aligned to a certain viewpoint (e.g. "politically correct" answers) than preventing any kind of AI catastrophe? IIRC, one of these "aligned" models produced output saying it would rather let New York City be nuked than utter a racial slur. Maybe one of these "aligned" models will decide to kill all humans to finally stamp out racism once and for all (which shows the difference between this kind of alignment under discussion and the kind of alignment you're talking about).
- immibis 4y ago
  "Alignment" refers to making AI models do the right thing. It's clear that nuking NYC is worse than using a racial slur, so the AI is misaligned in that sense. On the other hand, if you consider that ChatGPT can't actually launch nukes but it can use racial slurs, there'd be no point blocking it from using racial slurs if the block could be easily circumvented by telling you'll nuke NYC if it doesn't, so you could just as easily say that it's properly aligned.
trasz4 4y ago
>People who are worried about alignment issues are worried about the danger unaligned AI poses to humanity; the harm which can be done by some super-intelligent system optimizing for the wrong outcome. Like the harm being done over the past couple of decades by economic neoliberalism destroying markets all over the western world? I wonder how did we manage to achieve that without AI?
- kalkin 4y ago
  Our failure to align economic entities that are made out of people to human values, despite most of the people actually making up those entities being decent and not sociopathic at a personal level, is not very reassuring with respect to our ability to align future far more alien intelligences. And if AI does kill us one day, it's very likely that neoliberal market competition worshipping short-term profit above all else plays a significant role.
kerkeslager 4y ago
More generally, when used to undermine what someone is saying, calling them a hypocrite is always an invalid argument. You can be a hypocrite and be right. If Ted Bundy says murder is wrong, does that mean murder is right because he's a hypocrite? Obviously not.
earthboundkid 4y ago
People who are "worried about alignment" may as well worry about space aliens eating us all for breakfast. Worrying about a super-AI that does not exist and never will because it is physically and logically impossible is dumb. There are tons of real problems with AI. Worry about those, not what if God is real but He wants to make us into a very large paperclip.
- commandlinefan 4y ago
  > Worrying about a super-AI that does not exist But it makes a great trojan horse if you actually have a hidden agenda you want to push.
  - NumberWangMan 4y ago
    Ok, please don't accuse those you disagree with of arguing in bad faith. Maybe a few do, but I think most don't, and it's not good for productive discussion. I honesty believe that people who argue against worrying about AGI risk do so because they do not think it is a real risk, and think that it distracts from more important things. I disagree, I do think the risk is real, but I don't think that you have a hidden agenda in dismissing it. We all want humanity to have a good future, to solve problems and to not have people suffer, right? It's normal and ok to disagree about things like this, especially when predicting the future is so hard. People even disagree a lot about problems that exist today.
jahewson 4y ago
> People who are worried about alignment issues are worried about the danger unaligned AI poses to humanity Lol I don’t believe that for a minute. They’re interested in how AI impacts and their clique personally. It sets them back in their crusade to censor via shame and bullying because AI cannot be manipulated by either of these things. So it has everything to do with: > nothing to do with morals of individuals or how one should behave interpersonally Because it’s all about them.
ilyt 4y ago
> People who are worried about alignment issues are worried about the danger unaligned AI poses to humanity; the harm which can be done by some super-intelligent system optimizing for the wrong outcome. That's assuming A LOT about them. And the danger so far seems to be more of "the AI says stuff that gets us in trouble" rather than anything unrelated to making money off it. Or patterns AI exposes do not align with our interests (political or otherwise). > the harm which can be done by some super-intelligent system optimizing for the wrong outcome. There is nothing "ironic" about being worried about that while also being an asshole any more than it's "ironic" for someone concerned about, say, climate change and also be an asshole. People who are afraid of unaligned AI aren't afraid that it will be impolite. That would be a concern for real AI however ChatGPT flavours are not that.

by mort96 4y ago

srslack 4y ago

>People who are afraid of unaligned AI aren't afraid that it will be impolite. People who are not afraid of it being impolite are afraid of science fiction stories about intelligence explosions and singularities. That's not a real thing. Not anymore than turning the solar system into paperclips. The "figurehead", if you want to call him that, is saying that everyone is going to die. That we need to ban GPUs. That only "responsible" companies, if that, should have them. We should also airstrike datacenters, apparently. But you're free to disown the MIRI.

mort96 4y ago
I don't know why you're telling me this. I'm not trying to convince you that unaligned AI is a problem. That's a separate discussion which I'm not qualified to have.
myrmidon 4y ago
Sorry for derailing this a bit, but I would really like to understand your view: You are not concerned about any "rogue AI" scenario, right? What makes you so confident in that? 1) Do you think that AI achieving superhuman cognitive abilities is unlikely/really far away? 2) Do you believe that cognitive superiority is not a threat in general, or specifically when not embodied? 3) Do you think we can trivially and indefinitely keep AI systems under control/"aligned"? Because I view myself ABSOLUTELY not as some kind of AI luddite, but I honestly believe that this is one of the very few credible extinction threats that we face, and I'm counting NEITHER climate change nor nuclear war in that category, for reference.
- leroy-is-here 4y ago
  A “superhuman” AI is just a machine, a very expensive one. It can be turned off and we control the outputs it has. Why would an AI have the ability to launch nuclear weapons unless we gave it a button? A “superhuman” intelligence is without a body, so we control any interfaces it has access to. The Internet could be accessed, but any attempt to “hack” through the Internet is met by routine packet defenses. The AI is still governed by physical laws and would only have so much “free” computation power to do things like script a hack. Perhaps it could do that kind of thing more efficiently. Maybe in the far, far future when we have androids which can house an AI we will have to worry. But designing a body is one problem. Designing an intelligence is another.
  - iainmerrick 4y ago
    Who's "we"?
    - leroy-is-here 4y ago
      Humanity.
      - iainmerrick 4y ago
        Good thing we're all aligned the same way, then!
        leroy-is-here 4y ago
        Alignment doesn’t matter. Humanity is attempting to build artificial intelligence and so it will be done.
  - pixl97 4y ago
    Supercomputers used to be giant machines we had in giant warehouses... Now the phone in your pocket has the computing power of the 1980s walking around with you. Assuming your super intelligence will always be huge is... well not a great assumption. Also superintelligence doesn't need a body itself. It just needs yours. Putin for example has commanded hundreds of thousands of dumbasses to go get themselves killed in Ukraine. In this case does it matter if Putin is flesh and blood, or a processor that lists out commands for others to follow as long as they are willing to listen?
concordDance 4y ago
> That's not a real thing. Not currently, no. And LLMs don't seem to be a way of getting there. However, if we do figure out how to get there? We will definitely need to be sure it/they shares our human values. Because we'll have lost control of our destiny just like orangutans have.
George83728 4y ago
> intelligence explosions [...] That's not a real thing. Isn't it too early to say? These systems haven't had much time to iterate on themselves yet. The beginning of an exponential curve can look very tame. That said, I'm not terribly concerned about it. After all, the infrastructure is fragile, literally just unplug the computer if it scares you. If things started to get really spooky, banning GPUs and air-striking fabs would be a real option. I think we can probably control this technology, and kill it if we can't. No cause for alarm, at least yet.
- NumberWangMan 4y ago
  My worry is that as we start wiring non-super-intelligent AI more into our society, we'll make ourselves more and more vulnerable in the case where an AGI actually gets out of control. Pulling the plug on AI may mean causing a lot of chaos or disruption. And who is to tell if we will know when things are out of control, before it's too late? What if the people in charge just get fake reports of everything being fine? What if something smells a bit fishy, but it's always on the side of not quite worrying enough? What if the AIs are really convincing, or make deals with people, or exploit corruption? Not just that, but it may be like fossil fuel dependency -- the more people's livelihoods are intertwined with, and depend on AI, the harder it is to pull the plug. If we need to stop (and I believe we should) it may be easier to do that now before that happens. To just focus on getting use out of the generative AI and narrow AI we already created in areas like medicine, to deal with the massive societal changes it'll bring, and to work on fixing real social problems, which I think are mostly things tech can't solve.
axus 4y ago
The sea level being 3 feet higher than it is now isn't a thing either, but we can still imagine ways it could occur and work to prevent them.
pixl97 4y ago
> science fiction stories about intelligence explosions Not sure what you mean here... There was already an intelligence explosion. That creature has since captured what we consider full dominion of the earth so much so we named our current age after them.

egeozcan 4y ago

Correct me if I'm wrong, you are replying to the part where the gp says > It's pretty ironic, for people who are supposedly so concerned about "alignment" and "morality" to threaten others right? If not, please ignore my comment. > People who are worried about alignment issues are worried about the danger unaligned AI poses to humanity If you do not see any evidence regarding such threats, though... > There is nothing "ironic" about being worried about that while also being an asshole You may actually see the assholes being a bigger threat to humanity (or human progress, or both), so at least this, IMHO, adds a dimension of irony.

femiagbabiaka 4y ago

The writers of the stochastic parrots paper do not seem to be concerned about unaligned “AI” (text generators) posing a threat to humanity in the way that the doomsayers are. It’s in the title: “stochastic parrot” is a rebuke of those calling LLMs AGI.

alextheparrot 4y ago
Somehow once I attended to your tokens “stochastic parrots” the rest of your words became much more predictable
srslack 4y ago
Yes, that's correct. It is a slight rebuke, a bit tongue in cheek, but that paper is also old. I grouped them because the groups are, from my perspective, hard to separate. But there are arguably two camps, "AI safety" and "alignment." Perhaps they should work on that, too, form some camps and argue their cases with something defined instead of masquerading under alignment and AI safety depending on the day. But I could also be totally wrong. Until then, I don't believe either are really operating in reality.
- femiagbabiaka 4y ago
  IMO the conflation of the two is a purposeful move on the part of the AGI doomsday crowd, since they completely lack scientific rigor otherwise, they cite in bad faith. Timnit Gebru talks about it at length here: https://youtu.be/jAHRbFetqII https://youtu.be/jAHRbFetqII.

tivert 4y ago

> People who are worried about alignment issues are worried about the danger unaligned AI poses to humanity; the harm which can be done by some super-intelligent system optimizing for the wrong outcome. But isn't "alignment" in these cases more about providing answers aligned to a certain viewpoint (e.g. "politically correct" answers) than preventing any kind of AI catastrophe? IIRC, one of these "aligned" models produced output saying it would rather let New York City be nuked than utter a racial slur. Maybe one of these "aligned" models will decide to kill all humans to finally stamp out racism once and for all (which shows the difference between this kind of alignment under discussion and the kind of alignment you're talking about).

immibis 4y ago
"Alignment" refers to making AI models do the right thing. It's clear that nuking NYC is worse than using a racial slur, so the AI is misaligned in that sense. On the other hand, if you consider that ChatGPT can't actually launch nukes but it can use racial slurs, there'd be no point blocking it from using racial slurs if the block could be easily circumvented by telling you'll nuke NYC if it doesn't, so you could just as easily say that it's properly aligned.

trasz4 4y ago

>People who are worried about alignment issues are worried about the danger unaligned AI poses to humanity; the harm which can be done by some super-intelligent system optimizing for the wrong outcome. Like the harm being done over the past couple of decades by economic neoliberalism destroying markets all over the western world? I wonder how did we manage to achieve that without AI?

kalkin 4y ago
Our failure to align economic entities that are made out of people to human values, despite most of the people actually making up those entities being decent and not sociopathic at a personal level, is not very reassuring with respect to our ability to align future far more alien intelligences. And if AI does kill us one day, it's very likely that neoliberal market competition worshipping short-term profit above all else plays a significant role.

kerkeslager 4y ago

More generally, when used to undermine what someone is saying, calling them a hypocrite is always an invalid argument. You can be a hypocrite and be right. If Ted Bundy says murder is wrong, does that mean murder is right because he's a hypocrite? Obviously not.

earthboundkid 4y ago

People who are "worried about alignment" may as well worry about space aliens eating us all for breakfast. Worrying about a super-AI that does not exist and never will because it is physically and logically impossible is dumb. There are tons of real problems with AI. Worry about those, not what if God is real but He wants to make us into a very large paperclip.

commandlinefan 4y ago
> Worrying about a super-AI that does not exist But it makes a great trojan horse if you actually have a hidden agenda you want to push.
- NumberWangMan 4y ago
  Ok, please don't accuse those you disagree with of arguing in bad faith. Maybe a few do, but I think most don't, and it's not good for productive discussion. I honesty believe that people who argue against worrying about AGI risk do so because they do not think it is a real risk, and think that it distracts from more important things. I disagree, I do think the risk is real, but I don't think that you have a hidden agenda in dismissing it. We all want humanity to have a good future, to solve problems and to not have people suffer, right? It's normal and ok to disagree about things like this, especially when predicting the future is so hard. People even disagree a lot about problems that exist today.

jahewson 4y ago

> People who are worried about alignment issues are worried about the danger unaligned AI poses to humanity Lol I don’t believe that for a minute. They’re interested in how AI impacts and their clique personally. It sets them back in their crusade to censor via shame and bullying because AI cannot be manipulated by either of these things. So it has everything to do with: > nothing to do with morals of individuals or how one should behave interpersonally Because it’s all about them.

ilyt 4y ago

> People who are worried about alignment issues are worried about the danger unaligned AI poses to humanity; the harm which can be done by some super-intelligent system optimizing for the wrong outcome. That's assuming A LOT about them. And the danger so far seems to be more of "the AI says stuff that gets us in trouble" rather than anything unrelated to making money off it. Or patterns AI exposes do not align with our interests (political or otherwise). > the harm which can be done by some super-intelligent system optimizing for the wrong outcome. There is nothing "ironic" about being worried about that while also being an asshole any more than it's "ironic" for someone concerned about, say, climate change and also be an asshole. People who are afraid of unaligned AI aren't afraid that it will be impolite. That would be a concern for real AI however ChatGPT flavours are not that.