6 ms·
I don't understand this. How would being open source mitigate the risks of the technology?
by JohnFen 4y ago
I don't understand this. How would being open source mitigate the risks of the technology?
- SuoDuanDao 4y agoit would mitigate the risks of bad actors using it against everyone else. Open source technology would be weaponized, but not asymmetrically so.
- JohnFen 4y agoThat doesn't sound like mitigating the risks to me, though. Bad actors will still use it against everyone else. That everyone else can use it against everyone else as well doesn't seem to address this problem.
- SuoDuanDao 4y agoOne bad actor can lock down any attempt to counteract them. Four Billion bad actors will be in an arms race where everyone can decide for themselves what protections they want to enact. This is basically the free speech argument again. A few more cycles + targeting and it will be very easy to convince anyone of anything, barring a way to answer. Do we want one party with free, very convincing speech or eight billion? Personally I'd prefer eight billion. It improves the odds I'll be convinced of something good.
- JohnFen 4y agoI think both outcomes are potentially disastrous, to be honest. I can't decide which would be worse.
- idopmstuff 4y agoIs that good, though? I've seen a number of people compare AI to nukes, and I think that's sort of a good analogy. The thing is, it's really good that not anybody can make their own nukes. Instead, they're (mostly) in the hands of powerful countries who all have some motivation not to use them, because if they fire them they'll end up in a nuclear war that makes them less powerful. If terrorist groups had easy access to nukes, they'd likely be a lot more likely to fire them because they have less to lose (and more insane ideologies). With LLMs, I understand why people look very negatively on the prospect of them being controlled by major corporations - certainly there are huge issues with that. But if we believe that they're going to develop world-ending capabilities, I would rather they be in the hands of wealthy, powerful corporations that are strongly incentivized not to destroy the world, vs. being freely available to people who want to burn it all down. Mutually assured destruction works because the only people who are party to it are those who don't want to be destroyed. If everyone has access to world-ending tech, it only takes one person who doesn't care to end it all. In that sense, asymmetry is very good.
- suoduandao2 4y agoI think ultimately, we're going to have a dark age where the AIs are driving for a while. My prediction is that it'll last until society learns something we're overlooking about intelligence unlike ourselves. I prefer the open-source scenario, not because I'm sure it would be less pleasant but because I'm sure it wouldn't last as long.
- nonbirithm 4y agoTerrorists would have a field day if they had access to nuclear weapons. The only thing stopping them is that the barrier to independently developing nuclear weapons is so high that it's only practical for powerful nation-states to have them. In comparison it's trivial for both terrorists and powerful nation-states to download 60GB of weights and crunch numbers on a GPU cluster. Arguably the hardware requirements of the most sophisticated LLMs act as a similar barrier to widespread adoption even if they were open-source, but to nowhere near the same extent, and as with nukes that's just a happy coincidence of the current state of technology/known physical limitations than an intentional barrier to stopping their spread. Even as we speak there has been collective interest in discovering a way to decrease those hardware requirements (INT8/INT4 quantization) as much as possible so more people can run them for themselves, which shows no signs of abating. All I can say is we're lucky that physics prevents us from mass-producing ICBMs on a fast enough time schedule, because if it were possible we'd probably attempt to do so for the sake of trying.
- SuoDuanDao 4y agochatbots are not all that similar to ICBMs. Information warfare exists in an ecosystem where individuals have a huge amount of control over what gets in or out - defenses to malicious information can't be overwhelmed the way an ICBM overwhelms physical defenses, they have to be subverted. How hard they are to subvert is a function of prior information the target has had. Therefore, more asymmetry makes us more vulnerable.
- shrimp_emoji 4y agoYou're working with black box technologies beyond comprehensibility (which is its own problem), but at least you know the ingredients that went in the data stew and what the shenanigans people have been up to are.
- JohnFen 4y agoHow does that follow? The magic in the software isn't really the software as much as the training data. How does being able to see the source code give you any insight in terms of what data it is trained with, or what people are doing with it?
- seydor 4y agothe data is compiled by programs.
- sergeen 4y agoWhen a model is open-source the community can access and scrutinize its code and workings. This can help to detect potential safety issues and biases. Just think about what a mega-corporation like Microsoft, whose primary interests lie in accumulating capital and market control, can do with this technology that is essentially embedded into everything. Of course, there are potential benefits, but I'm personally more interested in the risks.
- seydor 4y agoIt wouldn't but it s auditable so we would know it wouldn't. We can't trust that openAI is not creating risks