4 ms·

Seriously? This is just silly. Everyone knows the barrier to terrorists using bio weapons is not specialist knowledge, but access to labs, equipment, reagents e

Seriously? This is just silly. Everyone knows the barrier to terrorists using bio weapons is not specialist knowledge, but access to labs, equipment, reagents etc. It's the whole Guttenberg's printing press argument. "Whoaa hold on now, what do you mean you want knowledge to be freely available to the vulgar masses?" The only difference with LLMs is that you do not have to search for this knowledge by yourself, you get a very much hallucination prone AI to tell you the answers. If we extend this argument further why don't we restrict access to public libraries, scientific research and neuter Google even more. And what about Wikipedia?

cubefox 3y ago
Neither Wikipedia nor public libraries allow instructions to make weapons of mass destruction.
- xcdzvyn 3y ago
  All of the information AI regurgitates is either already available online as part of its corpus (and therefore the AI plays no particular role in access to that information), or completely made up (which is likely to kill more terrorists than anyone else!) Reiterating other comments, terrorists can't make bioweapons because they lack the facilities and prerequisites, not because they're incompetent.
  - cubefox 3y ago
    Top AI researchers like Geoffrey Hinton say that large language models likely have an internal world model and aren't just stochastic parrots. Which means they can do more than just repeating strings from the training distribution. Facilities are a major hurdle for nuclear weapons. For bioweapons they are much less of a problem. The main constraint is competency.
    - junofan 3y ago
      I think you might want to take a look at some of the history here, and particularly the cyclical nature of the AI field for the past 50–60 years. It’s helpful to put what everyone’s saying in context.
    - riversflow 3y ago
      They have no way to test that model though. > The main constraint is competency. Oh right, anyone can be a chemist, it requires no skill, that why labs aren't a core part of the course work. Ai researchers are really good at telling other fields their work, that they have no experience in, is easy.
      - cubefox 3y ago
        Kevin Esvelt very much has experience. I posted a link to his Wikipedia page.
  - 0xDEAFBEAD 3y ago
    The "all the info is already online" argument is also an argument against LLMs in general. If you really believe that argument, you shouldn't care one way or another about LLM release. After all, the LLM doesn't tell you anything that's not on Google. Either the LLM is useful, in which case it could be useful to a terrorist, or it's useless, in which case you won't mind if access is restricted. Note: I'm not saying it will definitely be useful to a terrorist. I'm saying that companies have an obligation to show in advance that their open source LLM can't help a terrorist, before releasing it.
    - riversflow 3y ago
      > "all the info is already online" argument is also an argument against LLMs in general. No it isnt. That's like saying, "You can walk" is an argument against cars.
      - 0xDEAFBEAD 3y ago
        And cars have been pretty useful to terrorists: https://en.wikipedia.org/wiki/Vehicle-ramming_attack#Terrorism https://en.wikipedia.org/wiki/Vehicle-ramming_attack#Terrori... If LLMs are set to revolutionize industry after industry, why not the terrorism industry? Someone should be thinking about this beyond just "I don't see how LLMs would help a terrorist after 60 seconds of thought". Perhaps the overall cost/benefit is such that LLMs should still be open-source, similar to how we don't restrict cars -- my point is that it should be an informed decision. And we should also recognize that it's really hard to have this discussion in public. The best way to argue that LLMs could be used by terrorists is for me to give details of particular schemes for doing terrorism with LLMs, and I don't care to publish such schemes. [BTW, my basic mental model here is that terrorists are often not all that educated and we are terrifically lucky for that. I'm in favor of breakthrough tutoring technology in general, just not for bioweapons-adjacent knowledge. And I think bioweapons have much stronger potential for an outlier terrorist attack compared with cars.]
0xDEAFBEAD 3y ago
>Everyone knows the barrier to terrorists using bio weapons is not specialist knowledge, but access to labs, equipment, reagents etc. An LLM could help you get that access, or help you make do without it. >It's the whole Guttenberg's printing press argument. "Whoaa hold on now, what do you mean you want knowledge to be freely available to the vulgar masses?" We're fortunate that intelligent, educated people typically don't choose to become terrorists and criminals. Every generation of improved LLMs has the potential to expand the set of people who could construct bioweapons. It's true that technology is typically good and beneficial, but beware the parable of the turkey: https://www.forbes.com/sites/hillennevins/2021/11/24/the-parable-of-the-turkey/ https://www.forbes.com/sites/hillennevins/2021/11/24/the-par... A Thanksgiving turkey could have a wonderful life until late November when it gets slaughtered out of the blue. We can't just count on trends to continue indefinitely -- a famous example would be the 2008 financial crisis, before which people assumed that "housing prices always go up". It's just common sense to forecast the possibility of extreme risks and think about how to mitigate them. And yes, I favor across the board restrictions on information deemed sensitive. But people publishing open source LLMs should have an obligation to show that what they're releasing will not increase the likelihood of catastrophic risks. https://www.stat.berkeley.edu/~aldous/157/Papers/yudkowsky.pdf https://www.stat.berkeley.edu/~aldous/157/Papers/yudkowsky.p...

cubefox 3y ago

Neither Wikipedia nor public libraries allow instructions to make weapons of mass destruction.

xcdzvyn 3y ago
All of the information AI regurgitates is either already available online as part of its corpus (and therefore the AI plays no particular role in access to that information), or completely made up (which is likely to kill more terrorists than anyone else!) Reiterating other comments, terrorists can't make bioweapons because they lack the facilities and prerequisites, not because they're incompetent.
- cubefox 3y ago
  Top AI researchers like Geoffrey Hinton say that large language models likely have an internal world model and aren't just stochastic parrots. Which means they can do more than just repeating strings from the training distribution. Facilities are a major hurdle for nuclear weapons. For bioweapons they are much less of a problem. The main constraint is competency.
  - junofan 3y ago
    I think you might want to take a look at some of the history here, and particularly the cyclical nature of the AI field for the past 50–60 years. It’s helpful to put what everyone’s saying in context.
  - riversflow 3y ago
    They have no way to test that model though. > The main constraint is competency. Oh right, anyone can be a chemist, it requires no skill, that why labs aren't a core part of the course work. Ai researchers are really good at telling other fields their work, that they have no experience in, is easy.
    - cubefox 3y ago
      Kevin Esvelt very much has experience. I posted a link to his Wikipedia page.
- 0xDEAFBEAD 3y ago
  The "all the info is already online" argument is also an argument against LLMs in general. If you really believe that argument, you shouldn't care one way or another about LLM release. After all, the LLM doesn't tell you anything that's not on Google. Either the LLM is useful, in which case it could be useful to a terrorist, or it's useless, in which case you won't mind if access is restricted. Note: I'm not saying it will definitely be useful to a terrorist. I'm saying that companies have an obligation to show in advance that their open source LLM can't help a terrorist, before releasing it.
  - riversflow 3y ago
    > "all the info is already online" argument is also an argument against LLMs in general. No it isnt. That's like saying, "You can walk" is an argument against cars.
    - 0xDEAFBEAD 3y ago
      And cars have been pretty useful to terrorists: https://en.wikipedia.org/wiki/Vehicle-ramming_attack#Terrorism https://en.wikipedia.org/wiki/Vehicle-ramming_attack#Terrori... If LLMs are set to revolutionize industry after industry, why not the terrorism industry? Someone should be thinking about this beyond just "I don't see how LLMs would help a terrorist after 60 seconds of thought". Perhaps the overall cost/benefit is such that LLMs should still be open-source, similar to how we don't restrict cars -- my point is that it should be an informed decision. And we should also recognize that it's really hard to have this discussion in public. The best way to argue that LLMs could be used by terrorists is for me to give details of particular schemes for doing terrorism with LLMs, and I don't care to publish such schemes. [BTW, my basic mental model here is that terrorists are often not all that educated and we are terrifically lucky for that. I'm in favor of breakthrough tutoring technology in general, just not for bioweapons-adjacent knowledge. And I think bioweapons have much stronger potential for an outlier terrorist attack compared with cars.]

0xDEAFBEAD 3y ago

>Everyone knows the barrier to terrorists using bio weapons is not specialist knowledge, but access to labs, equipment, reagents etc. An LLM could help you get that access, or help you make do without it. >It's the whole Guttenberg's printing press argument. "Whoaa hold on now, what do you mean you want knowledge to be freely available to the vulgar masses?" We're fortunate that intelligent, educated people typically don't choose to become terrorists and criminals. Every generation of improved LLMs has the potential to expand the set of people who could construct bioweapons. It's true that technology is typically good and beneficial, but beware the parable of the turkey: https://www.forbes.com/sites/hillennevins/2021/11/24/the-parable-of-the-turkey/ https://www.forbes.com/sites/hillennevins/2021/11/24/the-par... A Thanksgiving turkey could have a wonderful life until late November when it gets slaughtered out of the blue. We can't just count on trends to continue indefinitely -- a famous example would be the 2008 financial crisis, before which people assumed that "housing prices always go up". It's just common sense to forecast the possibility of extreme risks and think about how to mitigate them. And yes, I favor across the board restrictions on information deemed sensitive. But people publishing open source LLMs should have an obligation to show that what they're releasing will not increase the likelihood of catastrophic risks. https://www.stat.berkeley.edu/~aldous/157/Papers/yudkowsky.pdf https://www.stat.berkeley.edu/~aldous/157/Papers/yudkowsky.p...