5 ms·

Very sad to see they did not make it open source. When you have a technology that has the potential to be a gateway for drug development, to the cures of new di

Very sad to see they did not make it open source. When you have a technology that has the potential to be a gateway for drug development, to the cures of new diseases, and instead you choose to make it closed, it is a very huge disservice to the community at large. Sure, release your own product alongside it, but making it closed source does not help the scientific community upon which all these innovations were built. Especially if you have lost a loved one to a disease which this technology will one day be able to create cures for, it is very disappointing.

falcor84 2y ago
The closer it gets to enabling full drug discovery, the closer it also gets to enabling bioterrorism. Taking it to the extreme, if they had the theory of everything, I don't think I'd want it to be made available to the whole world as it is today. On a related note, I highly recommend The Talos Principle 2, which really made me think about these questions.
- pythonguython 2y ago
  Any organization/country that has the ability to use a tool like this to create a bio weapon is already sophisticated enough to do bioterrorism today.
  - ramon156 2y ago
    Alright, but now picture this: it's now open to the masses, meaning an individual could probably even do it.
    - uptownfunk 2y ago
      The risks don't exceed what is already out there. If someone wants to do damage, especially in America, there are more than enough ways they can do it already. The technology should be made free. I also wonder how much the claims are being exaggerated and are marketing-speak vs. real results. Is there any benchmark for this that they have published?
      - bongodongobob 2y ago
        No. I love to be egalitarian as well but this AI thing really feels different. We didn't just invent a better plow or a more durable sword. We're working on making a better brain. I think social media shows us a pretty good slice of the average person and it's not great. Now imagine they can manipulate the smartest person in the world to do dangerous, dumb shit.
        MaxikCZ 2y ago
        >we didnt just invent a better plow But we invented metalworking. And if metal were for kings chairs only, we would still have no plows.
    - pythonguython 2y ago
      I hear you, but I don't think an individual can. If I gave you $20,000 and an open source 70% accurate protein folding model and told you to develop, mass produce, and develop a deployment mechanism for a highly infectious and deadly pathogen, I don't think you could make that happen. Nor do I think you could do it if you had a PhD in microbiology.
      - bongodongobob 2y ago
        right now
    - consumer451 2y ago
      You raise an extremely important point. It appears to me that most people do not understand the implications of your point. Organized terrorism by groups is actually extremely rare. What is much less rare are mass shootings in the USA, by deranged individuals. What would a psychopathic mass shooter type choose as a weapon if he not only had access to semi-automatic weapons, but now we added bio-weapons to the menu? It seems very clear to me that when creating custom viruses becomes high school level knowledge, and the tools can be charged on a credit card, nuclear weapons will be relegated to the second most likely way that our human civilization will end. I believe the two concepts being brought together here are the Law of Large Numbers, and the sudden ability for one single human to kill at least millions.
      - tsimionescu 2y ago
        > It seems very clear to me that when creating custom viruses becomes high school level knowledge That would be very bad indeed, but there is no path from AI to that. Making custom viruses is never going to be an easy task even if you had a magic machine that could explain the effects of adding any chemical to the mix. You still need to procure the chemicals and work with them in very careful ways, often for a long time, in a highly controlled environment. It's still biology lab work, even if you know exactly what you have to do. Also, bioweapons already exist and have been used in a few conflicts, even as recently as WWII. They're terrifying in many ways, but are not really comparable to the horror of nuclear weapons hitting major cities.
        consumer451 2y ago
        > Making custom viruses is never going to be an easy task even if you had a magic machine that could explain the effects of adding any chemical to the mix. You still need to procure the chemicals and work with them in very careful ways, often for a long time, in a highly controlled environment. It's still biology lab work, even if you know exactly what you have to do. You appear to be talking about today. I am referring to some point in the future. If you extrapolate our technological progress out to the future, it certainly seems possible, at some point.
    - tsimionescu 2y ago
      The problem of producing bio weapons is not computational, it's physical in nature. Even if predictions from these tools become 100% accurate and encompassed 100% of the chemistry, you still need to actually do the manual steps to breed the bioagents. And, very importantly, you need to do so without getting you and your co-workers deadly ill long before finishing the thing. Which requires extremely sophisticated machinery. Alternatively, you can go today in some of the poorer corners of the world, find some people with drug resistant tuberculosis, pay them a pittance to give you bodily fluids, and disperse those in a large crowd, say at a concert or similar. You'll get a good chunk of the effects of the worse possible bioterrorism.
      - hackernewds 2y ago
        These are very specific ideas you have..
        baq 2y ago
        These are ideas you should read and think about because intelligence agencies all over the world have been thinking about them for the past hundred years.
        BriggyDwiggs42 2y ago
        It’s very frustrating when people consider the possession of information equivalent to malice. It suggests that the right way to run society is to keep people stupid and harmless.
        baq 2y ago
        Harmless is enough. The smart ones will figure out that knowing shortens lives.
        FeepingCreature 2y ago
        Okay, I'll live in the harmless society, you go live in the harmful society.
    - TeMPOraL 2y ago
      Honestly, I wouldn't worry about bioterrorism as much as handling mishap. Stick new proteins into a bacteria the wrong way, don't wash hands thoroughly enough, and suddenly something is eating all the trees in the region, or whatnot. Designing an effective, lethal pathogen - fast enough to do damage, but slow enough to not burn itself out - is hard. Accidentally making something ecologically damaging is probably much simpler, and I imagine the future holds plenty such localized minor ecophagy[0] event. -- [0] - Yes, I totally just learned that term from https://en.wikipedia.org/wiki/Gray_goo https://en.wikipedia.org/wiki/Gray_goo a minute ago.
- LouisSayers 2y ago
  Why do you need AI for bioterrorism? There are plenty of well known biological organisms that can kill us today...
- BriggyDwiggs42 2y ago
  Oh please, like a terrorist cant fork over a couple bucks to do the bioterrorism. This excuse is utter bs, whether its applied to LLMs or to alphafold. The motivator is profit, not safety.
- datadeft 2y ago
  Why on Earth would any terrorist want to invest into this when you can just purchase guns and explosives much easier?
  - AlexCoventry 2y ago
    Because a pathogen could potentially sicken or kill many more people.
- deleted 2y ago
  [deleted]
robertlagrant 2y ago
> the community at large Which community? Not any I'm part of.
DrScientist 2y ago
I suspect https://www.isomorphiclabs.com/ https://www.isomorphiclabs.com/ is the reason. There are 3 basic ways to fund research. - Taxes - most academic research - begging - research charities - profits - companies like Google. Sometimes the lines get blurred - but I don't think you can expect Google to release as much of their work for free as people who are paid via central taxes.
- abecedarius 2y ago
  Worth noting that they did release the AlphaFold 2 weights after a while. Milking an expensive discovery for a limited period should be considered laudable, unless you think tax funding of all research would be awesome and it's just a weird anomaly how the org producing these results was a tiny heterodox startup very recently.
  - DrScientist 2y ago
    > Worth noting that they did release the AlphaFold 2 weights after a while. Yes - though I don't think Isomorphic labs existed at that point. Obviously the real reason AlphaFold was possible was the huge tax payer funded effort running over decades to generate a diverse, high quality 3D structure dataset. However that's why we put taxes into research - to spur innovation in a pre-competitive way - so that's fine. What's not fine is any benefiting company avoiding paying any tax back on resulting profits - that's just free riding - and many of the big tech companies are, in my view, guilty.
    - abecedarius 2y ago
      The data is a treasure. It was just as available to others, before and after.
      - DrScientist 2y ago
        Yep not knocking alphafold - Alphafold showed it could be done[1], and others have subsequently followed. However if all the tech companies hoover up all the profits and simultaneously avoid paying the appropriate level of taxes - then the cycle of innovation isn't sustainable. As well taxes paying for that pre-competitive data, they train the next generation of PhDs. [1] There were other groups making progress with DL based structure prediction before Alphafold, but alphafold was a leap forward.
DonsDiscountGas 2y ago
I'd be willing to bet that OpenFold has an implementation inside of a few months. https://openfold.io/ https://openfold.io/
Xeyz0r 2y ago
Openness and collaboration could have far-reaching implications for public health and well-being but there are lots of aspects of being open

falcor84 2y ago

The closer it gets to enabling full drug discovery, the closer it also gets to enabling bioterrorism. Taking it to the extreme, if they had the theory of everything, I don't think I'd want it to be made available to the whole world as it is today. On a related note, I highly recommend The Talos Principle 2, which really made me think about these questions.

pythonguython 2y ago
Any organization/country that has the ability to use a tool like this to create a bio weapon is already sophisticated enough to do bioterrorism today.
- ramon156 2y ago
  Alright, but now picture this: it's now open to the masses, meaning an individual could probably even do it.
  - uptownfunk 2y ago
    The risks don't exceed what is already out there. If someone wants to do damage, especially in America, there are more than enough ways they can do it already. The technology should be made free. I also wonder how much the claims are being exaggerated and are marketing-speak vs. real results. Is there any benchmark for this that they have published?
    - bongodongobob 2y ago
      No. I love to be egalitarian as well but this AI thing really feels different. We didn't just invent a better plow or a more durable sword. We're working on making a better brain. I think social media shows us a pretty good slice of the average person and it's not great. Now imagine they can manipulate the smartest person in the world to do dangerous, dumb shit.
      - MaxikCZ 2y ago
        >we didnt just invent a better plow But we invented metalworking. And if metal were for kings chairs only, we would still have no plows.
  - pythonguython 2y ago
    I hear you, but I don't think an individual can. If I gave you $20,000 and an open source 70% accurate protein folding model and told you to develop, mass produce, and develop a deployment mechanism for a highly infectious and deadly pathogen, I don't think you could make that happen. Nor do I think you could do it if you had a PhD in microbiology.
    - bongodongobob 2y ago
      right now
  - consumer451 2y ago
    You raise an extremely important point. It appears to me that most people do not understand the implications of your point. Organized terrorism by groups is actually extremely rare. What is much less rare are mass shootings in the USA, by deranged individuals. What would a psychopathic mass shooter type choose as a weapon if he not only had access to semi-automatic weapons, but now we added bio-weapons to the menu? It seems very clear to me that when creating custom viruses becomes high school level knowledge, and the tools can be charged on a credit card, nuclear weapons will be relegated to the second most likely way that our human civilization will end. I believe the two concepts being brought together here are the Law of Large Numbers, and the sudden ability for one single human to kill at least millions.
    - tsimionescu 2y ago
      > It seems very clear to me that when creating custom viruses becomes high school level knowledge That would be very bad indeed, but there is no path from AI to that. Making custom viruses is never going to be an easy task even if you had a magic machine that could explain the effects of adding any chemical to the mix. You still need to procure the chemicals and work with them in very careful ways, often for a long time, in a highly controlled environment. It's still biology lab work, even if you know exactly what you have to do. Also, bioweapons already exist and have been used in a few conflicts, even as recently as WWII. They're terrifying in many ways, but are not really comparable to the horror of nuclear weapons hitting major cities.
      - consumer451 2y ago
        > Making custom viruses is never going to be an easy task even if you had a magic machine that could explain the effects of adding any chemical to the mix. You still need to procure the chemicals and work with them in very careful ways, often for a long time, in a highly controlled environment. It's still biology lab work, even if you know exactly what you have to do. You appear to be talking about today. I am referring to some point in the future. If you extrapolate our technological progress out to the future, it certainly seems possible, at some point.
  - tsimionescu 2y ago
    The problem of producing bio weapons is not computational, it's physical in nature. Even if predictions from these tools become 100% accurate and encompassed 100% of the chemistry, you still need to actually do the manual steps to breed the bioagents. And, very importantly, you need to do so without getting you and your co-workers deadly ill long before finishing the thing. Which requires extremely sophisticated machinery. Alternatively, you can go today in some of the poorer corners of the world, find some people with drug resistant tuberculosis, pay them a pittance to give you bodily fluids, and disperse those in a large crowd, say at a concert or similar. You'll get a good chunk of the effects of the worse possible bioterrorism.
    - hackernewds 2y ago
      These are very specific ideas you have..
      - baq 2y ago
        These are ideas you should read and think about because intelligence agencies all over the world have been thinking about them for the past hundred years.
      - BriggyDwiggs42 2y ago
        It’s very frustrating when people consider the possession of information equivalent to malice. It suggests that the right way to run society is to keep people stupid and harmless.
        baq 2y ago
        Harmless is enough. The smart ones will figure out that knowing shortens lives.
        FeepingCreature 2y ago
        Okay, I'll live in the harmless society, you go live in the harmful society.
  - TeMPOraL 2y ago
    Honestly, I wouldn't worry about bioterrorism as much as handling mishap. Stick new proteins into a bacteria the wrong way, don't wash hands thoroughly enough, and suddenly something is eating all the trees in the region, or whatnot. Designing an effective, lethal pathogen - fast enough to do damage, but slow enough to not burn itself out - is hard. Accidentally making something ecologically damaging is probably much simpler, and I imagine the future holds plenty such localized minor ecophagy[0] event. -- [0] - Yes, I totally just learned that term from https://en.wikipedia.org/wiki/Gray_goo https://en.wikipedia.org/wiki/Gray_goo a minute ago.
LouisSayers 2y ago
Why do you need AI for bioterrorism? There are plenty of well known biological organisms that can kill us today...
BriggyDwiggs42 2y ago
Oh please, like a terrorist cant fork over a couple bucks to do the bioterrorism. This excuse is utter bs, whether its applied to LLMs or to alphafold. The motivator is profit, not safety.
datadeft 2y ago
Why on Earth would any terrorist want to invest into this when you can just purchase guns and explosives much easier?
- AlexCoventry 2y ago
  Because a pathogen could potentially sicken or kill many more people.
deleted 2y ago
[deleted]

robertlagrant 2y ago

> the community at large Which community? Not any I'm part of.

DrScientist 2y ago

I suspect https://www.isomorphiclabs.com/ https://www.isomorphiclabs.com/ is the reason. There are 3 basic ways to fund research. - Taxes - most academic research - begging - research charities - profits - companies like Google. Sometimes the lines get blurred - but I don't think you can expect Google to release as much of their work for free as people who are paid via central taxes.

abecedarius 2y ago
Worth noting that they did release the AlphaFold 2 weights after a while. Milking an expensive discovery for a limited period should be considered laudable, unless you think tax funding of all research would be awesome and it's just a weird anomaly how the org producing these results was a tiny heterodox startup very recently.
- DrScientist 2y ago
  > Worth noting that they did release the AlphaFold 2 weights after a while. Yes - though I don't think Isomorphic labs existed at that point. Obviously the real reason AlphaFold was possible was the huge tax payer funded effort running over decades to generate a diverse, high quality 3D structure dataset. However that's why we put taxes into research - to spur innovation in a pre-competitive way - so that's fine. What's not fine is any benefiting company avoiding paying any tax back on resulting profits - that's just free riding - and many of the big tech companies are, in my view, guilty.
  - abecedarius 2y ago
    The data is a treasure. It was just as available to others, before and after.
    - DrScientist 2y ago
      Yep not knocking alphafold - Alphafold showed it could be done[1], and others have subsequently followed. However if all the tech companies hoover up all the profits and simultaneously avoid paying the appropriate level of taxes - then the cycle of innovation isn't sustainable. As well taxes paying for that pre-competitive data, they train the next generation of PhDs. [1] There were other groups making progress with DL based structure prediction before Alphafold, but alphafold was a leap forward.

DonsDiscountGas 2y ago

I'd be willing to bet that OpenFold has an implementation inside of a few months. https://openfold.io/ https://openfold.io/

Xeyz0r 2y ago

Openness and collaboration could have far-reaching implications for public health and well-being but there are lots of aspects of being open