4 ms·

The closer it gets to enabling full drug discovery, the closer it also gets to enabling bioterrorism. Taking it to the extreme, if they had the theory of everyt

The closer it gets to enabling full drug discovery, the closer it also gets to enabling bioterrorism. Taking it to the extreme, if they had the theory of everything, I don't think I'd want it to be made available to the whole world as it is today. On a related note, I highly recommend The Talos Principle 2, which really made me think about these questions.

pythonguython 3y ago
Any organization/country that has the ability to use a tool like this to create a bio weapon is already sophisticated enough to do bioterrorism today.
- ramon156 3y ago
  Alright, but now picture this: it's now open to the masses, meaning an individual could probably even do it.
  - uptownfunk 3y ago
    The risks don't exceed what is already out there. If someone wants to do damage, especially in America, there are more than enough ways they can do it already. The technology should be made free. I also wonder how much the claims are being exaggerated and are marketing-speak vs. real results. Is there any benchmark for this that they have published?
    - bongodongobob 3y ago
      No. I love to be egalitarian as well but this AI thing really feels different. We didn't just invent a better plow or a more durable sword. We're working on making a better brain. I think social media shows us a pretty good slice of the average person and it's not great. Now imagine they can manipulate the smartest person in the world to do dangerous, dumb shit.
      - MaxikCZ 3y ago
        >we didnt just invent a better plow But we invented metalworking. And if metal were for kings chairs only, we would still have no plows.
  - pythonguython 3y ago
    I hear you, but I don't think an individual can. If I gave you $20,000 and an open source 70% accurate protein folding model and told you to develop, mass produce, and develop a deployment mechanism for a highly infectious and deadly pathogen, I don't think you could make that happen. Nor do I think you could do it if you had a PhD in microbiology.
    - bongodongobob 3y ago
      right now
  - consumer451 3y ago
    You raise an extremely important point. It appears to me that most people do not understand the implications of your point. Organized terrorism by groups is actually extremely rare. What is much less rare are mass shootings in the USA, by deranged individuals. What would a psychopathic mass shooter type choose as a weapon if he not only had access to semi-automatic weapons, but now we added bio-weapons to the menu? It seems very clear to me that when creating custom viruses becomes high school level knowledge, and the tools can be charged on a credit card, nuclear weapons will be relegated to the second most likely way that our human civilization will end. I believe the two concepts being brought together here are the Law of Large Numbers, and the sudden ability for one single human to kill at least millions.
    - tsimionescu 3y ago
      > It seems very clear to me that when creating custom viruses becomes high school level knowledge That would be very bad indeed, but there is no path from AI to that. Making custom viruses is never going to be an easy task even if you had a magic machine that could explain the effects of adding any chemical to the mix. You still need to procure the chemicals and work with them in very careful ways, often for a long time, in a highly controlled environment. It's still biology lab work, even if you know exactly what you have to do. Also, bioweapons already exist and have been used in a few conflicts, even as recently as WWII. They're terrifying in many ways, but are not really comparable to the horror of nuclear weapons hitting major cities.
      - consumer451 3y ago
        > Making custom viruses is never going to be an easy task even if you had a magic machine that could explain the effects of adding any chemical to the mix. You still need to procure the chemicals and work with them in very careful ways, often for a long time, in a highly controlled environment. It's still biology lab work, even if you know exactly what you have to do. You appear to be talking about today. I am referring to some point in the future. If you extrapolate our technological progress out to the future, it certainly seems possible, at some point.
  - tsimionescu 3y ago
    The problem of producing bio weapons is not computational, it's physical in nature. Even if predictions from these tools become 100% accurate and encompassed 100% of the chemistry, you still need to actually do the manual steps to breed the bioagents. And, very importantly, you need to do so without getting you and your co-workers deadly ill long before finishing the thing. Which requires extremely sophisticated machinery. Alternatively, you can go today in some of the poorer corners of the world, find some people with drug resistant tuberculosis, pay them a pittance to give you bodily fluids, and disperse those in a large crowd, say at a concert or similar. You'll get a good chunk of the effects of the worse possible bioterrorism.
    - hackernewds 3y ago
      These are very specific ideas you have..
      - baq 3y ago
        These are ideas you should read and think about because intelligence agencies all over the world have been thinking about them for the past hundred years.
      - BriggyDwiggs42 3y ago
        It’s very frustrating when people consider the possession of information equivalent to malice. It suggests that the right way to run society is to keep people stupid and harmless.
        baq 3y ago
        Harmless is enough. The smart ones will figure out that knowing shortens lives.
        FeepingCreature 3y ago
        Okay, I'll live in the harmless society, you go live in the harmful society.
  - TeMPOraL 3y ago
    Honestly, I wouldn't worry about bioterrorism as much as handling mishap. Stick new proteins into a bacteria the wrong way, don't wash hands thoroughly enough, and suddenly something is eating all the trees in the region, or whatnot. Designing an effective, lethal pathogen - fast enough to do damage, but slow enough to not burn itself out - is hard. Accidentally making something ecologically damaging is probably much simpler, and I imagine the future holds plenty such localized minor ecophagy[0] event. -- [0] - Yes, I totally just learned that term from https://en.wikipedia.org/wiki/Gray_goo https://en.wikipedia.org/wiki/Gray_goo a minute ago.
LouisSayers 3y ago
Why do you need AI for bioterrorism? There are plenty of well known biological organisms that can kill us today...
BriggyDwiggs42 3y ago
Oh please, like a terrorist cant fork over a couple bucks to do the bioterrorism. This excuse is utter bs, whether its applied to LLMs or to alphafold. The motivator is profit, not safety.
datadeft 3y ago
Why on Earth would any terrorist want to invest into this when you can just purchase guns and explosives much easier?
- AlexCoventry 3y ago
  Because a pathogen could potentially sicken or kill many more people.
deleted 3y ago
[deleted]

pythonguython 3y ago

Any organization/country that has the ability to use a tool like this to create a bio weapon is already sophisticated enough to do bioterrorism today.

ramon156 3y ago
Alright, but now picture this: it's now open to the masses, meaning an individual could probably even do it.
- uptownfunk 3y ago
  The risks don't exceed what is already out there. If someone wants to do damage, especially in America, there are more than enough ways they can do it already. The technology should be made free. I also wonder how much the claims are being exaggerated and are marketing-speak vs. real results. Is there any benchmark for this that they have published?
  - bongodongobob 3y ago
    No. I love to be egalitarian as well but this AI thing really feels different. We didn't just invent a better plow or a more durable sword. We're working on making a better brain. I think social media shows us a pretty good slice of the average person and it's not great. Now imagine they can manipulate the smartest person in the world to do dangerous, dumb shit.
    - MaxikCZ 3y ago
      >we didnt just invent a better plow But we invented metalworking. And if metal were for kings chairs only, we would still have no plows.
- pythonguython 3y ago
  I hear you, but I don't think an individual can. If I gave you $20,000 and an open source 70% accurate protein folding model and told you to develop, mass produce, and develop a deployment mechanism for a highly infectious and deadly pathogen, I don't think you could make that happen. Nor do I think you could do it if you had a PhD in microbiology.
  - bongodongobob 3y ago
    right now
- consumer451 3y ago
  You raise an extremely important point. It appears to me that most people do not understand the implications of your point. Organized terrorism by groups is actually extremely rare. What is much less rare are mass shootings in the USA, by deranged individuals. What would a psychopathic mass shooter type choose as a weapon if he not only had access to semi-automatic weapons, but now we added bio-weapons to the menu? It seems very clear to me that when creating custom viruses becomes high school level knowledge, and the tools can be charged on a credit card, nuclear weapons will be relegated to the second most likely way that our human civilization will end. I believe the two concepts being brought together here are the Law of Large Numbers, and the sudden ability for one single human to kill at least millions.
  - tsimionescu 3y ago
    > It seems very clear to me that when creating custom viruses becomes high school level knowledge That would be very bad indeed, but there is no path from AI to that. Making custom viruses is never going to be an easy task even if you had a magic machine that could explain the effects of adding any chemical to the mix. You still need to procure the chemicals and work with them in very careful ways, often for a long time, in a highly controlled environment. It's still biology lab work, even if you know exactly what you have to do. Also, bioweapons already exist and have been used in a few conflicts, even as recently as WWII. They're terrifying in many ways, but are not really comparable to the horror of nuclear weapons hitting major cities.
    - consumer451 3y ago
      > Making custom viruses is never going to be an easy task even if you had a magic machine that could explain the effects of adding any chemical to the mix. You still need to procure the chemicals and work with them in very careful ways, often for a long time, in a highly controlled environment. It's still biology lab work, even if you know exactly what you have to do. You appear to be talking about today. I am referring to some point in the future. If you extrapolate our technological progress out to the future, it certainly seems possible, at some point.
- tsimionescu 3y ago
  The problem of producing bio weapons is not computational, it's physical in nature. Even if predictions from these tools become 100% accurate and encompassed 100% of the chemistry, you still need to actually do the manual steps to breed the bioagents. And, very importantly, you need to do so without getting you and your co-workers deadly ill long before finishing the thing. Which requires extremely sophisticated machinery. Alternatively, you can go today in some of the poorer corners of the world, find some people with drug resistant tuberculosis, pay them a pittance to give you bodily fluids, and disperse those in a large crowd, say at a concert or similar. You'll get a good chunk of the effects of the worse possible bioterrorism.
  - hackernewds 3y ago
    These are very specific ideas you have..
    - baq 3y ago
      These are ideas you should read and think about because intelligence agencies all over the world have been thinking about them for the past hundred years.
    - BriggyDwiggs42 3y ago
      It’s very frustrating when people consider the possession of information equivalent to malice. It suggests that the right way to run society is to keep people stupid and harmless.
      - baq 3y ago
        Harmless is enough. The smart ones will figure out that knowing shortens lives.
      - FeepingCreature 3y ago
        Okay, I'll live in the harmless society, you go live in the harmful society.
- TeMPOraL 3y ago
  Honestly, I wouldn't worry about bioterrorism as much as handling mishap. Stick new proteins into a bacteria the wrong way, don't wash hands thoroughly enough, and suddenly something is eating all the trees in the region, or whatnot. Designing an effective, lethal pathogen - fast enough to do damage, but slow enough to not burn itself out - is hard. Accidentally making something ecologically damaging is probably much simpler, and I imagine the future holds plenty such localized minor ecophagy[0] event. -- [0] - Yes, I totally just learned that term from https://en.wikipedia.org/wiki/Gray_goo https://en.wikipedia.org/wiki/Gray_goo a minute ago.

LouisSayers 3y ago

Why do you need AI for bioterrorism? There are plenty of well known biological organisms that can kill us today...

BriggyDwiggs42 3y ago

Oh please, like a terrorist cant fork over a couple bucks to do the bioterrorism. This excuse is utter bs, whether its applied to LLMs or to alphafold. The motivator is profit, not safety.

datadeft 3y ago

Why on Earth would any terrorist want to invest into this when you can just purchase guns and explosives much easier?

AlexCoventry 3y ago
Because a pathogen could potentially sicken or kill many more people.

deleted 3y ago

[deleted]