5 ms·

it would mitigate the risks of bad actors using it against everyone else. Open source technology would be weaponized, but not asymmetrically so.

JohnFen 4y ago
That doesn't sound like mitigating the risks to me, though. Bad actors will still use it against everyone else. That everyone else can use it against everyone else as well doesn't seem to address this problem.
- SuoDuanDao 4y ago
  One bad actor can lock down any attempt to counteract them. Four Billion bad actors will be in an arms race where everyone can decide for themselves what protections they want to enact. This is basically the free speech argument again. A few more cycles + targeting and it will be very easy to convince anyone of anything, barring a way to answer. Do we want one party with free, very convincing speech or eight billion? Personally I'd prefer eight billion. It improves the odds I'll be convinced of something good.
  - JohnFen 4y ago
    I think both outcomes are potentially disastrous, to be honest. I can't decide which would be worse.
idopmstuff 4y ago
Is that good, though? I've seen a number of people compare AI to nukes, and I think that's sort of a good analogy. The thing is, it's really good that not anybody can make their own nukes. Instead, they're (mostly) in the hands of powerful countries who all have some motivation not to use them, because if they fire them they'll end up in a nuclear war that makes them less powerful. If terrorist groups had easy access to nukes, they'd likely be a lot more likely to fire them because they have less to lose (and more insane ideologies). With LLMs, I understand why people look very negatively on the prospect of them being controlled by major corporations - certainly there are huge issues with that. But if we believe that they're going to develop world-ending capabilities, I would rather they be in the hands of wealthy, powerful corporations that are strongly incentivized not to destroy the world, vs. being freely available to people who want to burn it all down. Mutually assured destruction works because the only people who are party to it are those who don't want to be destroyed. If everyone has access to world-ending tech, it only takes one person who doesn't care to end it all. In that sense, asymmetry is very good.
- suoduandao2 4y ago
  I think ultimately, we're going to have a dark age where the AIs are driving for a while. My prediction is that it'll last until society learns something we're overlooking about intelligence unlike ourselves. I prefer the open-source scenario, not because I'm sure it would be less pleasant but because I'm sure it wouldn't last as long.
nonbirithm 4y ago
Terrorists would have a field day if they had access to nuclear weapons. The only thing stopping them is that the barrier to independently developing nuclear weapons is so high that it's only practical for powerful nation-states to have them. In comparison it's trivial for both terrorists and powerful nation-states to download 60GB of weights and crunch numbers on a GPU cluster. Arguably the hardware requirements of the most sophisticated LLMs act as a similar barrier to widespread adoption even if they were open-source, but to nowhere near the same extent, and as with nukes that's just a happy coincidence of the current state of technology/known physical limitations than an intentional barrier to stopping their spread. Even as we speak there has been collective interest in discovering a way to decrease those hardware requirements (INT8/INT4 quantization) as much as possible so more people can run them for themselves, which shows no signs of abating. All I can say is we're lucky that physics prevents us from mass-producing ICBMs on a fast enough time schedule, because if it were possible we'd probably attempt to do so for the sake of trying.
- SuoDuanDao 4y ago
  chatbots are not all that similar to ICBMs. Information warfare exists in an ecosystem where individuals have a huge amount of control over what gets in or out - defenses to malicious information can't be overwhelmed the way an ICBM overwhelms physical defenses, they have to be subverted. How hard they are to subvert is a function of prior information the target has had. Therefore, more asymmetry makes us more vulnerable.

JohnFen 4y ago

That doesn't sound like mitigating the risks to me, though. Bad actors will still use it against everyone else. That everyone else can use it against everyone else as well doesn't seem to address this problem.

SuoDuanDao 4y ago
One bad actor can lock down any attempt to counteract them. Four Billion bad actors will be in an arms race where everyone can decide for themselves what protections they want to enact. This is basically the free speech argument again. A few more cycles + targeting and it will be very easy to convince anyone of anything, barring a way to answer. Do we want one party with free, very convincing speech or eight billion? Personally I'd prefer eight billion. It improves the odds I'll be convinced of something good.
- JohnFen 4y ago
  I think both outcomes are potentially disastrous, to be honest. I can't decide which would be worse.

idopmstuff 4y ago

Is that good, though? I've seen a number of people compare AI to nukes, and I think that's sort of a good analogy. The thing is, it's really good that not anybody can make their own nukes. Instead, they're (mostly) in the hands of powerful countries who all have some motivation not to use them, because if they fire them they'll end up in a nuclear war that makes them less powerful. If terrorist groups had easy access to nukes, they'd likely be a lot more likely to fire them because they have less to lose (and more insane ideologies). With LLMs, I understand why people look very negatively on the prospect of them being controlled by major corporations - certainly there are huge issues with that. But if we believe that they're going to develop world-ending capabilities, I would rather they be in the hands of wealthy, powerful corporations that are strongly incentivized not to destroy the world, vs. being freely available to people who want to burn it all down. Mutually assured destruction works because the only people who are party to it are those who don't want to be destroyed. If everyone has access to world-ending tech, it only takes one person who doesn't care to end it all. In that sense, asymmetry is very good.

suoduandao2 4y ago
I think ultimately, we're going to have a dark age where the AIs are driving for a while. My prediction is that it'll last until society learns something we're overlooking about intelligence unlike ourselves. I prefer the open-source scenario, not because I'm sure it would be less pleasant but because I'm sure it wouldn't last as long.

nonbirithm 4y ago

Terrorists would have a field day if they had access to nuclear weapons. The only thing stopping them is that the barrier to independently developing nuclear weapons is so high that it's only practical for powerful nation-states to have them. In comparison it's trivial for both terrorists and powerful nation-states to download 60GB of weights and crunch numbers on a GPU cluster. Arguably the hardware requirements of the most sophisticated LLMs act as a similar barrier to widespread adoption even if they were open-source, but to nowhere near the same extent, and as with nukes that's just a happy coincidence of the current state of technology/known physical limitations than an intentional barrier to stopping their spread. Even as we speak there has been collective interest in discovering a way to decrease those hardware requirements (INT8/INT4 quantization) as much as possible so more people can run them for themselves, which shows no signs of abating. All I can say is we're lucky that physics prevents us from mass-producing ICBMs on a fast enough time schedule, because if it were possible we'd probably attempt to do so for the sake of trying.

SuoDuanDao 4y ago
chatbots are not all that similar to ICBMs. Information warfare exists in an ecosystem where individuals have a huge amount of control over what gets in or out - defenses to malicious information can't be overwhelmed the way an ICBM overwhelms physical defenses, they have to be subverted. How hard they are to subvert is a function of prior information the target has had. Therefore, more asymmetry makes us more vulnerable.