4 ms·

Proliferation of more advanced AIs without any control would increase the power of some malicious groups far beyond they currently have. This paper explores on

Proliferation of more advanced AIs without any control would increase the power of some malicious groups far beyond they currently have. This paper explores one such danger and there are other papers which show it's possible to use LLM to aid in designing new toxins and biological weapons. The Operational Risks of AI in Large-Scale Biological Attacks https://www.rand.org/pubs/research_reports/RRA2977-1.html https://www.rand.org/pubs/research_reports/RRA2977-1.html? An example of such an event: https://en.wikipedia.org/wiki/Tokyo_subway_sarin_attack https://en.wikipedia.org/wiki/Tokyo_subway_sarin_attack How do you propose we deal with this sort of harm if more powerful AIs with no limit and control proliferate in the wild? . Note: Both sides of the OpenAI rift care deeply about AI Safety. They just follow different approaches. See more details here: https://news.ycombinator.com/item?id=38376263 https://news.ycombinator.com/item?id=38376263

kvgr 3y ago
If somebody wanted to do a biological attack, there is probably not much stopping them even now.
- nopinsight 3y ago
  The expertise to produce the substance itself is quite rare so it's hard to carry it out unnoticed. AI could make it much easier to develop it in one's basement.
  - swells34 3y ago
    Huh, you'd think all you need are some books on the subject and some fairly generic lab equipment. Not sure what a neural net trained on Internet dumps can add to that? The information has to be in the training data for the AI to be aware of it, correct?
    - nopinsight 3y ago
      GPT-4 is likely trained on some data not publicly available as well. There's also a distinction between trying to follow some broad textbook information and getting detailed feedback from an advanced conversational AI with vision and more knowledge than in a few textbooks/articles in real time.
  - DebtDeflation 3y ago
    The Tokyo Subway attack you referenced above happened in 1995 and didn't require AI. The information required can be found on the internet or in college textbooks. I suppose an "AI" in the sense of a chatbot can make it easier by summarizing these sources, but no one sufficiently motivated (and evil) would need that technology to do it.
nickpp 3y ago
> Proliferation of more advanced AIs without any control would increase the power of some malicious groups far beyond they currently have. Don't forget that it would also increase the power of the good guys. Any technology in history (starting with fire) had good and bad uses but overall the good outweighed the bad in every case. And considering that our default fate is extinction (by Sun's death if no other means) - we need all the good we can get to avoid that.
- nopinsight 3y ago
  > Don't forget that it would also increase the power of the good guys. In a free society, preventing and undoing a bioweapon attack or a pandemic is much harder than committing it. > And considering that our default fate is extinction (by Sun's death if no other means) - we need all the good we can get to avoid that. “In the long run we are all dead" -- Keynes. But an AGI will likely emerge in the next 5 to 20 years (Geoffrey Hinton said the same) and we'd rather not be dead too soon.
  - fallingknife 3y ago
    > In a free society, preventing and undoing a bioweapon attack or a pandemic is much harder than committing it. Is it? The hypothetical technology that allows someone to create an execute a bio weapon must have an understanding of molecular machinery that can also be uses to create a treatment.
    - NumberWangMan 3y ago
      I would say...not necessarily. The technology that lets someone create a gun does not give the ability to make bulletproof armor or the ability to treat life-threatening gunshot wounds. Or take nerve gases, as another example. It's entirely possible that we can learn how to make horrible pathogens without an equivalent means of curing them. Yes, there is probably some overlap in our understanding of biology for disease and cure, but it is a mistake to assume that they will balance each other out.
  - nickpp 3y ago
miraculixx 3y ago
Such attacks cannot be stopped by outlawing technology.

kvgr 3y ago

If somebody wanted to do a biological attack, there is probably not much stopping them even now.

nopinsight 3y ago
The expertise to produce the substance itself is quite rare so it's hard to carry it out unnoticed. AI could make it much easier to develop it in one's basement.
- swells34 3y ago
  Huh, you'd think all you need are some books on the subject and some fairly generic lab equipment. Not sure what a neural net trained on Internet dumps can add to that? The information has to be in the training data for the AI to be aware of it, correct?
  - nopinsight 3y ago
    GPT-4 is likely trained on some data not publicly available as well. There's also a distinction between trying to follow some broad textbook information and getting detailed feedback from an advanced conversational AI with vision and more knowledge than in a few textbooks/articles in real time.
- DebtDeflation 3y ago
  The Tokyo Subway attack you referenced above happened in 1995 and didn't require AI. The information required can be found on the internet or in college textbooks. I suppose an "AI" in the sense of a chatbot can make it easier by summarizing these sources, but no one sufficiently motivated (and evil) would need that technology to do it.

nickpp 3y ago

> Proliferation of more advanced AIs without any control would increase the power of some malicious groups far beyond they currently have. Don't forget that it would also increase the power of the good guys. Any technology in history (starting with fire) had good and bad uses but overall the good outweighed the bad in every case. And considering that our default fate is extinction (by Sun's death if no other means) - we need all the good we can get to avoid that.

nopinsight 3y ago
> Don't forget that it would also increase the power of the good guys. In a free society, preventing and undoing a bioweapon attack or a pandemic is much harder than committing it. > And considering that our default fate is extinction (by Sun's death if no other means) - we need all the good we can get to avoid that. “In the long run we are all dead" -- Keynes. But an AGI will likely emerge in the next 5 to 20 years (Geoffrey Hinton said the same) and we'd rather not be dead too soon.
- fallingknife 3y ago
  > In a free society, preventing and undoing a bioweapon attack or a pandemic is much harder than committing it. Is it? The hypothetical technology that allows someone to create an execute a bio weapon must have an understanding of molecular machinery that can also be uses to create a treatment.
  - NumberWangMan 3y ago
    I would say...not necessarily. The technology that lets someone create a gun does not give the ability to make bulletproof armor or the ability to treat life-threatening gunshot wounds. Or take nerve gases, as another example. It's entirely possible that we can learn how to make horrible pathogens without an equivalent means of curing them. Yes, there is probably some overlap in our understanding of biology for disease and cure, but it is a mistake to assume that they will balance each other out.
- nickpp 3y ago

miraculixx 3y ago

Such attacks cannot be stopped by outlawing technology.