5 ms·

As far as 'AI risk' stuff goes, I expect this kind of shit to actually kill people.

vouaobrasil 3y ago
Yes, if AI can do this, expect AI to also find diseases that are perfect bioweapons...such as a long-lasting common cold that merely weakens you for months, and is extremely contagious. Perfect for weakening an entire country before invasion, without even a hint that it actually is a bioweapon until it's too late.
- pawelmurias 3y ago
  Weapons that attack the entire planet are not perfect.
  - anshorei 3y ago
    They will be when our robot overlords want to get rid of us. :-)
    - ChatGTP 3y ago
      This assumes robots would want to be friend with each other, because robots. They'd probably wipe each other out very efficiently too and they would probably be able to do it without the use of weapons, it would just be cyber warfare.
  - hypertele-Xii 3y ago
    No weapon in the history of mankind has ever been perfect.
    - pawelmurias 3y ago
      Loads of things can be used as inefficient weapons, for example we could be worried about rouge states developing methods of dropping dead cows from zeppelins.
      - hypertele-Xii 3y ago
        You jest, but launching dead cows from catapults was indeed a medieval siege tactic.
Baeocystin 3y ago
How? Not snark, genuine question. What do you see about this form of active molecule search that makes it uniquely dangerous?
- c_crank 3y ago
  It seems very close to resurrecting ancient diseases and protein strands 'just because we can'.
astrange 3y ago
We have medical trials for a reason. It doesn't matter what the drug discovery process is as long as you test them after discovering them.
- tasogare 3y ago
  [dead]
- pixl97 3y ago
  I'm assuming you mean 'We' as the US. Assuming everyone holds high standards may be a mistake.
  - astrange 3y ago
    Am not aware of any non-US countries having issues with this either. The US is actually too strict, which is why we don't have any of the good sunscreens.
- jjk166 3y ago
  One could imagine a sufficiently advanced AI coming up with drugs which are designed to appear harmless in drug trials despite still being dangerous, either because it legitimately wants to murder us or because its learning function will be incredibly biased towards passing drug trials. I mean, we already sometimes do that (see the Thalidomide Tradgedy), but presumably AI can do it faster and more efficiently in ways that we would struggle to anticipate.
- klyrs 3y ago
  [flagged]
  - astrange 3y ago
    A whole virus worth of mRNA would be a lot of mRNA (though, a prion wouldn't be) and I think we'd notice it happening.
- ggm 3y ago
  Agree. Personally I'm strong on "AI has real risks", which are mostly inflated beliefs in what it tells us. The protocols for drug testing are so high AI (if it works) is most likely to reduce prescreening delay, and if it had a risk it would be ignoring novel forms which don't conform to training, not high lethality: the drug test regimen deals with that already.
shrimp_emoji 3y ago
Open source genomes of engineered super viruses combined with garage-grade nucleotide synthesizers will with more probability than anything else I can think of. Or, heck, maybe the super virus escapes from a BSL4 lab on accident before the garage phase. :D (There's been precedent, and I'm not alluding to COVID.)
- XorNot 3y ago
  I love how the Longtermism people won't shut up about this without having one single clue what they're actually talking about. As though virus particles spontaneously pop into existence when you mix the right DNA strands together (a process that gets wildly expensive beyond a couple hundred basepairs and is destroyed if the proteins on your fingers get anywhere near it). That's notwithstanding the fact that you can't just turn a dial labelled "lethality" to produce a gene sequence.
  - c_crank 3y ago
    The expense seems to have gone down enough in recent years that there are too many sketchy biolabs mixing a lot of base pairs together. Or doing other kinds of things to cross breed these bugs.
- bulla 3y ago
  What precedents?
  - echelon 3y ago
    This is a good list on BSL incidents: https://en.wikipedia.org/wiki/List_of_laboratory_biosecurity_incidents https://en.wikipedia.org/wiki/List_of_laboratory_biosecurity...
  - somenameforme 3y ago
    Lots. I think one of the only arguments that needs to be made against things like gain-of-function research is this. [1] That's an incomprehensive list various announced biosecurity incidents. Just since 2000 we've had lab incidents, including leaks, with: anthrax, west nile virus, SARS, COVID (not implying China - there was a confirmed COVID leak in Taiwan), ebola, tuberculosis, dengue, smallpox, zika, polio, and more. And they're happening all throughout the world. That includes the US, China, Russia, Japan, Germany, Australia, UK, South Korea, Hungary, France, Taiwan, Netherlands, and more. Incidents specified as coming from BSL-4 labs include ebola and SARS, though the BSL level is not specified at all for most incidents. And I would take that is an extremely incomprehensive list given that many incidents are likely going to be tucked away or classified. There's playing with fire, and then there's this... which increasingly more feels like standing around a fire and seeing what happens if you start dumping kerosene into it, all in the name of firefighting - of course. [1] - https://en.wikipedia.org/wiki/List_of_laboratory_biosecurity_incidents https://en.wikipedia.org/wiki/List_of_laboratory_biosecurity...

Yes, if AI can do this, expect AI to also find diseases that are perfect bioweapons...such as a long-lasting common cold that merely weakens you for months, and is extremely contagious. Perfect for weakening an entire country before invasion, without even a hint that it actually is a bioweapon until it's too late.

pawelmurias 3y ago
Weapons that attack the entire planet are not perfect.
- anshorei 3y ago
  They will be when our robot overlords want to get rid of us. :-)
  - ChatGTP 3y ago
    This assumes robots would want to be friend with each other, because robots. They'd probably wipe each other out very efficiently too and they would probably be able to do it without the use of weapons, it would just be cyber warfare.
- hypertele-Xii 3y ago
  No weapon in the history of mankind has ever been perfect.
  - pawelmurias 3y ago
    Loads of things can be used as inefficient weapons, for example we could be worried about rouge states developing methods of dropping dead cows from zeppelins.
    - hypertele-Xii 3y ago
      You jest, but launching dead cows from catapults was indeed a medieval siege tactic.

Baeocystin 3y ago

How? Not snark, genuine question. What do you see about this form of active molecule search that makes it uniquely dangerous?

c_crank 3y ago
It seems very close to resurrecting ancient diseases and protein strands 'just because we can'.

astrange 3y ago

We have medical trials for a reason. It doesn't matter what the drug discovery process is as long as you test them after discovering them.

tasogare 3y ago
[dead]
pixl97 3y ago
I'm assuming you mean 'We' as the US. Assuming everyone holds high standards may be a mistake.
- astrange 3y ago
  Am not aware of any non-US countries having issues with this either. The US is actually too strict, which is why we don't have any of the good sunscreens.
jjk166 3y ago
One could imagine a sufficiently advanced AI coming up with drugs which are designed to appear harmless in drug trials despite still being dangerous, either because it legitimately wants to murder us or because its learning function will be incredibly biased towards passing drug trials. I mean, we already sometimes do that (see the Thalidomide Tradgedy), but presumably AI can do it faster and more efficiently in ways that we would struggle to anticipate.
klyrs 3y ago
[flagged]
- astrange 3y ago
  A whole virus worth of mRNA would be a lot of mRNA (though, a prion wouldn't be) and I think we'd notice it happening.
ggm 3y ago
Agree. Personally I'm strong on "AI has real risks", which are mostly inflated beliefs in what it tells us. The protocols for drug testing are so high AI (if it works) is most likely to reduce prescreening delay, and if it had a risk it would be ignoring novel forms which don't conform to training, not high lethality: the drug test regimen deals with that already.

shrimp_emoji 3y ago

Open source genomes of engineered super viruses combined with garage-grade nucleotide synthesizers will with more probability than anything else I can think of. Or, heck, maybe the super virus escapes from a BSL4 lab on accident before the garage phase. :D (There's been precedent, and I'm not alluding to COVID.)

XorNot 3y ago
I love how the Longtermism people won't shut up about this without having one single clue what they're actually talking about. As though virus particles spontaneously pop into existence when you mix the right DNA strands together (a process that gets wildly expensive beyond a couple hundred basepairs and is destroyed if the proteins on your fingers get anywhere near it). That's notwithstanding the fact that you can't just turn a dial labelled "lethality" to produce a gene sequence.
- c_crank 3y ago
  The expense seems to have gone down enough in recent years that there are too many sketchy biolabs mixing a lot of base pairs together. Or doing other kinds of things to cross breed these bugs.
bulla 3y ago
What precedents?
- echelon 3y ago
  This is a good list on BSL incidents: https://en.wikipedia.org/wiki/List_of_laboratory_biosecurity_incidents https://en.wikipedia.org/wiki/List_of_laboratory_biosecurity...
- somenameforme 3y ago
  Lots. I think one of the only arguments that needs to be made against things like gain-of-function research is this. [1] That's an incomprehensive list various announced biosecurity incidents. Just since 2000 we've had lab incidents, including leaks, with: anthrax, west nile virus, SARS, COVID (not implying China - there was a confirmed COVID leak in Taiwan), ebola, tuberculosis, dengue, smallpox, zika, polio, and more. And they're happening all throughout the world. That includes the US, China, Russia, Japan, Germany, Australia, UK, South Korea, Hungary, France, Taiwan, Netherlands, and more. Incidents specified as coming from BSL-4 labs include ebola and SARS, though the BSL level is not specified at all for most incidents. And I would take that is an extremely incomprehensive list given that many incidents are likely going to be tucked away or classified. There's playing with fire, and then there's this... which increasingly more feels like standing around a fire and seeing what happens if you start dumping kerosene into it, all in the name of firefighting - of course. [1] - https://en.wikipedia.org/wiki/List_of_laboratory_biosecurity_incidents https://en.wikipedia.org/wiki/List_of_laboratory_biosecurity...