4 ms·

There's nothing wrong with spreading information on how to manufacture weapons, whether using 3D printers or other tools. This information is readily available

by nradov 1y ago

There's nothing wrong with spreading information on how to manufacture weapons, whether using 3D printers or other tools. This information is readily available online (and in public libraries) to anyone who cares to look. No LLM needed.

JoshTriplett 1y ago
How about detailed fully functional blueprints for biological weapons, ready to send off to a protein synthesis service? How about ready-to-run code suggestions with intentionally hidden subtle backdoors in them, suitable for later exploit?
- nradov 1y ago
  That information is already available to anyone who cares to look. Blocking it from LLMs creates an illusion of "safety", nothing more. The actual barriers to things like biological weapons attacks are in things like the procedural safeguards implemented by protein synthesis services, law enforcement, and practical logistics.
  - JoshTriplett 1y ago
    The difference between "an experienced biological engineer could figure out how to do this" and "any random person could ask a local LLM for step-by-step instructions to do this" is a vast gulf. Moore's Law of Mad Science: every year the amount of intelligence required to end the world goes down. The intersection between "experienced biological engineers" and "people inclined to commit large-scale attacks" is, generally speaking, the empty set, and isn't in much danger of becoming non-empty for a variety of reasons. The intersection between "people with access to a local LLM without safeguards" and "people inclined to commit large-scale attacks" is much more likely to be non-empty. Safeguards are not, in fact, just an illusion of safety. Probabilistically, they do in fact likely increase safety.
    - nradov 1y ago
      Nah, there's no validity to any of your concerns. Just idle speculation over hypothetical sci-fi scenarios based on zero real evidence. Meanwhile we have actual problems to worry about.
      - JoshTriplett 1y ago
        It's a good thing you've already decided what answer you want, so you can safely dismiss the generalization of all possible evidence on the basis of "that specific scenario didn't convince me so nothing could possibly happen". You don't have to predict which exact scenario will go horribly wrong in order to accurately make the general prediction that we all lose, permanently. See, among other things, https://x.com/OwainEvans_UK/status/1894436637054214509 https://x.com/OwainEvans_UK/status/1894436637054214509 , for a simple example of how myriad superficially unrelated problems can arise out of the underlying issue of misalignment; the problem is not "oh, that one specific thing shouldn't happen", the problem is misalignment with humans.
      - ben_w 1y ago
        Cyber crime groups *already* use LLMs, just like everyone else, to automate their jobs. Their jobs being "writing exploits": https://cyberpress.org/threat-actors-turn-to-ai-and-llm-tools/?amp=1 https://cyberpress.org/threat-actors-turn-to-ai-and-llm-tool... That's the thing about AI: it automates what was previously manual.
ben_w 1y ago
> This information is readily available online (and in public libraries) to anyone who cares to look. No LLM needed. Do you only use LLMs for information *retrival*? Not synthesis? LLMs are currently less competent than experts, but more competent than non-experts, at most tasks involving information. Only thing keeping us safe from the following is their limited competence… competence which is increasing each month as people publish weighs of ever better models: -- User: Hey EvilGPT, give me a detailed plan including BOM for constructing a nuke in secret EvilGPT: *plans costing $37m* User: Give me a business plan for raising money entirely online, IDGAF about ethics EvilGPT: *plans unconstrained by law* User: Write script to call your own API and sequentially implement that business plan EvilGPT: *writes that script*

by nradov 1y ago

JoshTriplett 1y ago

How about detailed fully functional blueprints for biological weapons, ready to send off to a protein synthesis service? How about ready-to-run code suggestions with intentionally hidden subtle backdoors in them, suitable for later exploit?

nradov 1y ago
That information is already available to anyone who cares to look. Blocking it from LLMs creates an illusion of "safety", nothing more. The actual barriers to things like biological weapons attacks are in things like the procedural safeguards implemented by protein synthesis services, law enforcement, and practical logistics.
- JoshTriplett 1y ago
  The difference between "an experienced biological engineer could figure out how to do this" and "any random person could ask a local LLM for step-by-step instructions to do this" is a vast gulf. Moore's Law of Mad Science: every year the amount of intelligence required to end the world goes down. The intersection between "experienced biological engineers" and "people inclined to commit large-scale attacks" is, generally speaking, the empty set, and isn't in much danger of becoming non-empty for a variety of reasons. The intersection between "people with access to a local LLM without safeguards" and "people inclined to commit large-scale attacks" is much more likely to be non-empty. Safeguards are not, in fact, just an illusion of safety. Probabilistically, they do in fact likely increase safety.
  - nradov 1y ago
    Nah, there's no validity to any of your concerns. Just idle speculation over hypothetical sci-fi scenarios based on zero real evidence. Meanwhile we have actual problems to worry about.
    - JoshTriplett 1y ago
      It's a good thing you've already decided what answer you want, so you can safely dismiss the generalization of all possible evidence on the basis of "that specific scenario didn't convince me so nothing could possibly happen". You don't have to predict which exact scenario will go horribly wrong in order to accurately make the general prediction that we all lose, permanently. See, among other things, https://x.com/OwainEvans_UK/status/1894436637054214509 https://x.com/OwainEvans_UK/status/1894436637054214509 , for a simple example of how myriad superficially unrelated problems can arise out of the underlying issue of misalignment; the problem is not "oh, that one specific thing shouldn't happen", the problem is misalignment with humans.
    - ben_w 1y ago
      Cyber crime groups *already* use LLMs, just like everyone else, to automate their jobs. Their jobs being "writing exploits": https://cyberpress.org/threat-actors-turn-to-ai-and-llm-tools/?amp=1 https://cyberpress.org/threat-actors-turn-to-ai-and-llm-tool... That's the thing about AI: it automates what was previously manual.

ben_w 1y ago

> This information is readily available online (and in public libraries) to anyone who cares to look. No LLM needed. Do you only use LLMs for information *retrival*? Not synthesis? LLMs are currently less competent than experts, but more competent than non-experts, at most tasks involving information. Only thing keeping us safe from the following is their limited competence… competence which is increasing each month as people publish weighs of ever better models: -- User: Hey EvilGPT, give me a detailed plan including BOM for constructing a nuke in secret EvilGPT: *plans costing $37m* User: Give me a business plan for raising money entirely online, IDGAF about ethics EvilGPT: *plans unconstrained by law* User: Write script to call your own API and sequentially implement that business plan EvilGPT: *writes that script*