7 ms·

The difference between "an experienced biological engineer could figure out how to do this" and "any random person could ask a local LLM for step-by-step instru

The difference between "an experienced biological engineer could figure out how to do this" and "any random person could ask a local LLM for step-by-step instructions to do this" is a vast gulf. Moore's Law of Mad Science: every year the amount of intelligence required to end the world goes down. The intersection between "experienced biological engineers" and "people inclined to commit large-scale attacks" is, generally speaking, the empty set, and isn't in much danger of becoming non-empty for a variety of reasons. The intersection between "people with access to a local LLM without safeguards" and "people inclined to commit large-scale attacks" is much more likely to be non-empty. Safeguards are not, in fact, just an illusion of safety. Probabilistically, they do in fact likely increase safety.

nradov 1y ago
Nah, there's no validity to any of your concerns. Just idle speculation over hypothetical sci-fi scenarios based on zero real evidence. Meanwhile we have actual problems to worry about.
- JoshTriplett 1y ago
  It's a good thing you've already decided what answer you want, so you can safely dismiss the generalization of all possible evidence on the basis of "that specific scenario didn't convince me so nothing could possibly happen". You don't have to predict which exact scenario will go horribly wrong in order to accurately make the general prediction that we all lose, permanently. See, among other things, https://x.com/OwainEvans_UK/status/1894436637054214509 https://x.com/OwainEvans_UK/status/1894436637054214509 , for a simple example of how myriad superficially unrelated problems can arise out of the underlying issue of misalignment; the problem is not "oh, that one specific thing shouldn't happen", the problem is misalignment with humans.
- ben_w 1y ago
  Cyber crime groups *already* use LLMs, just like everyone else, to automate their jobs. Their jobs being "writing exploits": https://cyberpress.org/threat-actors-turn-to-ai-and-llm-tools/?amp=1 https://cyberpress.org/threat-actors-turn-to-ai-and-llm-tool... That's the thing about AI: it automates what was previously manual.

nradov 1y ago

Nah, there's no validity to any of your concerns. Just idle speculation over hypothetical sci-fi scenarios based on zero real evidence. Meanwhile we have actual problems to worry about.

JoshTriplett 1y ago
It's a good thing you've already decided what answer you want, so you can safely dismiss the generalization of all possible evidence on the basis of "that specific scenario didn't convince me so nothing could possibly happen". You don't have to predict which exact scenario will go horribly wrong in order to accurately make the general prediction that we all lose, permanently. See, among other things, https://x.com/OwainEvans_UK/status/1894436637054214509 https://x.com/OwainEvans_UK/status/1894436637054214509 , for a simple example of how myriad superficially unrelated problems can arise out of the underlying issue of misalignment; the problem is not "oh, that one specific thing shouldn't happen", the problem is misalignment with humans.
ben_w 1y ago
Cyber crime groups *already* use LLMs, just like everyone else, to automate their jobs. Their jobs being "writing exploits": https://cyberpress.org/threat-actors-turn-to-ai-and-llm-tools/?amp=1 https://cyberpress.org/threat-actors-turn-to-ai-and-llm-tool... That's the thing about AI: it automates what was previously manual.