4 ms·

It's a mistake to think that if something is a danger on the grounds of data privacy and economic reasons, then it's not an existential risk worth investigating

It's a mistake to think that if something is a danger on the grounds of data privacy and economic reasons, then it's not an existential risk worth investigating. Why not both?

__loam 3y ago
I think the existential risk argument is completely secondary to the social harms this technology is causing (and has been causing for some time even before generative models). Emphasizing it over mundane issues will lead to draconian regulatory regimes that fail to address the actual harms caused by this technology. The only things I can think of that make this an existential risk is if you're moronic enough to attach it to systems that are already an existential risk like letting gpt run your nuclear launch system or making bioweaponsGpt. Otherwise, if you're spinning fantasy that these things are self aware, you either have something to gain from people perceiving it that way like Sam Altman or you're an idiot like Yudkowsky.
- persnickety 3y ago
  You can't dismiss something as a non-risk unless you prove that the risk is zero. I bet dollars against peanuts that you can't even prove it on your own terms. Is the chance that someone will attach it to a nuclear launch or a bioweapon system zero? Not to even mention your or anyone else's ability to prove it today that it has no potential to become self-aware. Faced proved and hypothetical risks, I'd rather solve the former and investigate the latter than ignore any.
  - __loam 3y ago
    Investigate it, sure. Invest just as many resources into it though? I'm not saying the risk is completely zero, it's just an irrelevant and tiny quantity compared the actual problems this stuff is already causing. Rogue waves sinking massive shipping vessels in the middle of the ocean is a non-zero risk for example, but the probability is so low that it's not really a consideration for most shipping companies. Do you guys know how ai is going to become self aware? Or are we assuming that throwing enough compute at the problem will somehow lead to the singularity just because? That is, frankly, magical thinking and not really worth thinking about when compared to the absolutely massive abuses of the companies building these things.
    - persnickety 3y ago
      I agree that resource allocation is hard, but we're talking unknown unknowns here. There's precedent here that's being extrapolated: approximately no one expected ChatGPT ten years ago. We don't know whether there are stepwise improvements at any point in the future, so it's extremely hard to estimate the risk. Estimating it as "irrelevant" might end up just as accurate as predicting in 2000 that computers could not reach chatgpt levels of language mastery. It doesn't require an assumption, just allowing the possibility that a similar breakthrough in computing power and techniques might happen. In comparison, pretty much everyone (?) agrees about "abuses by the companies building those things", so there's no danger of that getting left neglected.
  - AnimalMuppet 3y ago
    Don't buy insurance that costs more than the risk does. Don't take cures that are worse than the disease. There's lots of non-zero risks that we just roll with, because the risk is less than the cost of the remediation. So your "prove the risk is zero" is the wrong standard. > Is the chance that someone will attach it to a nuclear launch or a bioweapon system zero? No. Was the chance that someone would attach an expert system to a nuclear launch zero? How about just ordinary buggy software? How about perfectly-implemented software that implements insane command-and-control doctrine? You seem to be demanding standards for AI that we haven't applied to anything else. > Not to even mention your or anyone else's ability to prove it today that it has no potential to become self-aware. That's going to be hard without an agreed-on definition for "self-aware". How do you suggest that we prove that it can't do something that we can't even define? And now I'm going to talk out of the other side of my mouth. It's a good thing that I think that the risk of it becoming self-aware are very close to zero, because I also think that if it can become self-aware, it's already too late to stop, at least if GPT is close (say, only one major breakthrough away). I mean, let's say the US and the EU decide to stop all research. Is any company going to secretly keep going in their own labs? (For that matter, will the Pentagon?) Even if they stop, will China? India? Brazil? Even if you are absolutely convinced that totally stopping is the correct answer, can you get buy-in from the entire rest of the world? No, you can't. So whatever the potential for harm is, we're going to see it. If you really think that it has a significant possibility of massive harm, then your best bet is to think about mitigations rather than prevention.

It's a mistake to think that if something is a danger on the grounds of data privacy and economic reasons, then it's not an existential risk worth investigating. Why not both?

__loam 3y ago

I think the existential risk argument is completely secondary to the social harms this technology is causing (and has been causing for some time even before generative models). Emphasizing it over mundane issues will lead to draconian regulatory regimes that fail to address the actual harms caused by this technology. The only things I can think of that make this an existential risk is if you're moronic enough to attach it to systems that are already an existential risk like letting gpt run your nuclear launch system or making bioweaponsGpt. Otherwise, if you're spinning fantasy that these things are self aware, you either have something to gain from people perceiving it that way like Sam Altman or you're an idiot like Yudkowsky.

persnickety 3y ago
You can't dismiss something as a non-risk unless you prove that the risk is zero. I bet dollars against peanuts that you can't even prove it on your own terms. Is the chance that someone will attach it to a nuclear launch or a bioweapon system zero? Not to even mention your or anyone else's ability to prove it today that it has no potential to become self-aware. Faced proved and hypothetical risks, I'd rather solve the former and investigate the latter than ignore any.
- __loam 3y ago
  Investigate it, sure. Invest just as many resources into it though? I'm not saying the risk is completely zero, it's just an irrelevant and tiny quantity compared the actual problems this stuff is already causing. Rogue waves sinking massive shipping vessels in the middle of the ocean is a non-zero risk for example, but the probability is so low that it's not really a consideration for most shipping companies. Do you guys know how ai is going to become self aware? Or are we assuming that throwing enough compute at the problem will somehow lead to the singularity just because? That is, frankly, magical thinking and not really worth thinking about when compared to the absolutely massive abuses of the companies building these things.
  - persnickety 3y ago
    I agree that resource allocation is hard, but we're talking unknown unknowns here. There's precedent here that's being extrapolated: approximately no one expected ChatGPT ten years ago. We don't know whether there are stepwise improvements at any point in the future, so it's extremely hard to estimate the risk. Estimating it as "irrelevant" might end up just as accurate as predicting in 2000 that computers could not reach chatgpt levels of language mastery. It doesn't require an assumption, just allowing the possibility that a similar breakthrough in computing power and techniques might happen. In comparison, pretty much everyone (?) agrees about "abuses by the companies building those things", so there's no danger of that getting left neglected.
- AnimalMuppet 3y ago
  Don't buy insurance that costs more than the risk does. Don't take cures that are worse than the disease. There's lots of non-zero risks that we just roll with, because the risk is less than the cost of the remediation. So your "prove the risk is zero" is the wrong standard. > Is the chance that someone will attach it to a nuclear launch or a bioweapon system zero? No. Was the chance that someone would attach an expert system to a nuclear launch zero? How about just ordinary buggy software? How about perfectly-implemented software that implements insane command-and-control doctrine? You seem to be demanding standards for AI that we haven't applied to anything else. > Not to even mention your or anyone else's ability to prove it today that it has no potential to become self-aware. That's going to be hard without an agreed-on definition for "self-aware". How do you suggest that we prove that it can't do something that we can't even define? And now I'm going to talk out of the other side of my mouth. It's a good thing that I think that the risk of it becoming self-aware are very close to zero, because I also think that if it can become self-aware, it's already too late to stop, at least if GPT is close (say, only one major breakthrough away). I mean, let's say the US and the EU decide to stop all research. Is any company going to secretly keep going in their own labs? (For that matter, will the Pentagon?) Even if they stop, will China? India? Brazil? Even if you are absolutely convinced that totally stopping is the correct answer, can you get buy-in from the entire rest of the world? No, you can't. So whatever the potential for harm is, we're going to see it. If you really think that it has a significant possibility of massive harm, then your best bet is to think about mitigations rather than prevention.