5 ms·

>how to kill everyone and it patiently and expertly walks them through how to create a global pandemic. There are thousands of people around the globe working

>how to kill everyone and it patiently and expertly walks them through how to create a global pandemic. There are thousands of people around the globe working in labs every day that can do this.

laverya 3y ago
And how many of them are radical religious nutjobs? I think it is plausible that "can spend decades learning a scientific craft" and "wants to kill everyone on the planet" are mutually exclusive personality traits.
- shoubidouwah 3y ago
  there is no OR gates in biology, only mutable distributions. the probability might be small for the two traits to appear, the overlap will occur with some frequency over billions of people. Also, and more importantly, the first condition is pretty heavily biological, the second is more of a social consequence. Just take a genius, give him internet and a healthy dollop of trauma, push the mysanthropism to 11 and watch the fireworks.
- TeMPOraL 3y ago
  Wasn't there a study that showed there are disproportionally many engineers among the more violent "radical religious nutjobs"? The hypothesis I heard for this was that STEM people are used to complex arguments and more abstract reasoning, so while they may be harder to initially convince, when they do get convinced, they're more likely to follow the new beliefs to their logical conclusions...
jefftk 3y ago
Which is very worrying! That's part of why I left big tech to work on detecting novel pandemics. [1] Luckily, the people capable of killing everyone seem not to have wanted to so far. But if the bar keeps lowering, at some point it will be accessible to "humanity is a cancer" folks and I do think we'll see serious efforts to wipe out humanity. [1] https://www.jefftk.com/p/computational-approaches-to-pathogen-detection https://www.jefftk.com/p/computational-approaches-to-pathoge...
- jebarker 3y ago
  Who are the "humanity is a cancer" folk? I thought this was a trope used to discredit environmentalists?
  - jefftk 3y ago
    Most people think continued existence of humanity is good, which includes environmentalists. But there are some people who either think (a) avoiding suffering is overwhelmingly important, to the extent that a universe without life is ideal or (b) the world would be better off without humans because of the vast destruction we cause. I'm not claiming this is a common view, but it doesn't have to be a common view to result in destruction if destruction gets sufficiently straightforward. For some writing in this direction, see https://longtermrisk.org/risks-of-astronomical-future-suffering/ https://longtermrisk.org/risks-of-astronomical-future-suffer... which argues that as suffering-focused people they should not try to kill everyone primarily because they are unlikely to succeed and https://philarchive.org/archive/KNUTWD https://philarchive.org/archive/KNUTWD ("Third, the negative utilitarian can argue that losing what currently exists on Earth would not be much of a loss, because of the following very plausible observation: overall, sentient beings on Earth fare terribly badly. The situation is not terrible for every single individual, but it is terrible when all individuals are considered. We can divide most sentient beings on Earth into the three categories of humans, non-human animals in captivity, and wild non-human animals, and deal with them in turn. ...")
    - jebarker 3y ago
      This doesn't really answer my question. These are arguments for why threat actors might hold the belief that "humanity is cancer", but it doesn't actually provide evidence that this is a credible threat in the real world.
      - jefftk 3y ago
        The two people I cited at the end are examples of philosophers who think a lifeless world is preferable to the status quo and to futures we are likely to get. These aren't arguments for why someone might hold the belief, but evidence that some people do seriously hold the belief. (Another example here would be https://en.m.wikipedia.org/wiki/Aum_Shinrikyo https://en.m.wikipedia.org/wiki/Aum_Shinrikyo)
bumby 3y ago
How does that number compare to the number of people (experts and laypeople) who have access to the mentioned AI models? Risk = severity x likelihood I think the OP's point was that AI increases the likelihood by dramatically increasing the access to that level of knowledge.
- edgyquant 3y ago
  So it’s open and shared knowledge you think is bad? Good to know.
  - bumby 3y ago
    Please go review the HN guidelines: https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html And to address the snarky mischaracterization, like with most things in life "it depends." As a general rule, I'm in favor of democratizing most resources, to include information. But there are caveats. I don't think, for example, non-anonymized health information should be open knowledge. Also, from the simple risk equation above, as the severity of consequence or the likelihood of misuse go up, there should probably be additional mitigations in place if we care about managing risk.
JoshTriplett 3y ago
There is a massive difference between "there are a few thousand people in the world who might have the specialized knowledge to kill billions" and "anyone with a handful of GPUs or access to a handful of cloud systems could have a conversation with LatestAIUnlimited-900T and make a few specialized mail orders". The Venn diagram of "people who could arrange to kill most of humanity without being stopped" and "people who want to kill most of humanity" is thankfully the empty set. If the former set expands to millions of people, that may not remain true.
- ipaddr 3y ago
  But someone could ask the AI how to stop it.
  - kaibee 3y ago
    It tends to be a lot easier to destroy than create, because entropy sucks like that.
- sterlind 3y ago
  there's a much easier way to create a global pandemic: wait for the next Ebola outbreak, hop on a plane, get infected, fly to NYC and spread your bodily fluids everywhere. I was (pleasantly) surprised that ISIL didn't try this. and making a pandemic virus is considerably more involved than making "a few specialized mail orders." and if it becomes easier in the future, far better to lock down the mail orders than the knowledge, no?
  - NoGravitas 3y ago
    Thankfully, Ebola is not a good candidate for a global pandemic for a number of reasons. It is much easier to avoid a virus spread by bodily fluid contact than one which is airborne, and Ebola tends to kill its hosts rather quickly. I'm more excited about SARS-CoV-5 (Captain Trips).
- mrguyorama 3y ago
  Generating bioweapons is not a trivial task even if you know exactly "how to do it". Ask any biologist how easy it is to grow X your very first attempt. Then remember buying starter cultures for most bioweapons isn't exactly easy and something anyone is allowed to do. Even an omniscient AI cannot overcome a skill issue

>how to kill everyone and it patiently and expertly walks them through how to create a global pandemic. There are thousands of people around the globe working in labs every day that can do this.

laverya 3y ago

And how many of them are radical religious nutjobs? I think it is plausible that "can spend decades learning a scientific craft" and "wants to kill everyone on the planet" are mutually exclusive personality traits.

shoubidouwah 3y ago
there is no OR gates in biology, only mutable distributions. the probability might be small for the two traits to appear, the overlap will occur with some frequency over billions of people. Also, and more importantly, the first condition is pretty heavily biological, the second is more of a social consequence. Just take a genius, give him internet and a healthy dollop of trauma, push the mysanthropism to 11 and watch the fireworks.
TeMPOraL 3y ago
Wasn't there a study that showed there are disproportionally many engineers among the more violent "radical religious nutjobs"? The hypothesis I heard for this was that STEM people are used to complex arguments and more abstract reasoning, so while they may be harder to initially convince, when they do get convinced, they're more likely to follow the new beliefs to their logical conclusions...

jefftk 3y ago

Which is very worrying! That's part of why I left big tech to work on detecting novel pandemics. [1] Luckily, the people capable of killing everyone seem not to have wanted to so far. But if the bar keeps lowering, at some point it will be accessible to "humanity is a cancer" folks and I do think we'll see serious efforts to wipe out humanity. [1] https://www.jefftk.com/p/computational-approaches-to-pathogen-detection https://www.jefftk.com/p/computational-approaches-to-pathoge...

jebarker 3y ago
Who are the "humanity is a cancer" folk? I thought this was a trope used to discredit environmentalists?
- jefftk 3y ago
  Most people think continued existence of humanity is good, which includes environmentalists. But there are some people who either think (a) avoiding suffering is overwhelmingly important, to the extent that a universe without life is ideal or (b) the world would be better off without humans because of the vast destruction we cause. I'm not claiming this is a common view, but it doesn't have to be a common view to result in destruction if destruction gets sufficiently straightforward. For some writing in this direction, see https://longtermrisk.org/risks-of-astronomical-future-suffering/ https://longtermrisk.org/risks-of-astronomical-future-suffer... which argues that as suffering-focused people they should not try to kill everyone primarily because they are unlikely to succeed and https://philarchive.org/archive/KNUTWD https://philarchive.org/archive/KNUTWD ("Third, the negative utilitarian can argue that losing what currently exists on Earth would not be much of a loss, because of the following very plausible observation: overall, sentient beings on Earth fare terribly badly. The situation is not terrible for every single individual, but it is terrible when all individuals are considered. We can divide most sentient beings on Earth into the three categories of humans, non-human animals in captivity, and wild non-human animals, and deal with them in turn. ...")
  - jebarker 3y ago
    This doesn't really answer my question. These are arguments for why threat actors might hold the belief that "humanity is cancer", but it doesn't actually provide evidence that this is a credible threat in the real world.
    - jefftk 3y ago
      The two people I cited at the end are examples of philosophers who think a lifeless world is preferable to the status quo and to futures we are likely to get. These aren't arguments for why someone might hold the belief, but evidence that some people do seriously hold the belief. (Another example here would be https://en.m.wikipedia.org/wiki/Aum_Shinrikyo https://en.m.wikipedia.org/wiki/Aum_Shinrikyo)

bumby 3y ago

How does that number compare to the number of people (experts and laypeople) who have access to the mentioned AI models? Risk = severity x likelihood I think the OP's point was that AI increases the likelihood by dramatically increasing the access to that level of knowledge.

edgyquant 3y ago
So it’s open and shared knowledge you think is bad? Good to know.
- bumby 3y ago
  Please go review the HN guidelines: https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html And to address the snarky mischaracterization, like with most things in life "it depends." As a general rule, I'm in favor of democratizing most resources, to include information. But there are caveats. I don't think, for example, non-anonymized health information should be open knowledge. Also, from the simple risk equation above, as the severity of consequence or the likelihood of misuse go up, there should probably be additional mitigations in place if we care about managing risk.

JoshTriplett 3y ago

There is a massive difference between "there are a few thousand people in the world who might have the specialized knowledge to kill billions" and "anyone with a handful of GPUs or access to a handful of cloud systems could have a conversation with LatestAIUnlimited-900T and make a few specialized mail orders". The Venn diagram of "people who could arrange to kill most of humanity without being stopped" and "people who want to kill most of humanity" is thankfully the empty set. If the former set expands to millions of people, that may not remain true.

ipaddr 3y ago
But someone could ask the AI how to stop it.
- kaibee 3y ago
  It tends to be a lot easier to destroy than create, because entropy sucks like that.
sterlind 3y ago
there's a much easier way to create a global pandemic: wait for the next Ebola outbreak, hop on a plane, get infected, fly to NYC and spread your bodily fluids everywhere. I was (pleasantly) surprised that ISIL didn't try this. and making a pandemic virus is considerably more involved than making "a few specialized mail orders." and if it becomes easier in the future, far better to lock down the mail orders than the knowledge, no?
- NoGravitas 3y ago
  Thankfully, Ebola is not a good candidate for a global pandemic for a number of reasons. It is much easier to avoid a virus spread by bodily fluid contact than one which is airborne, and Ebola tends to kill its hosts rather quickly. I'm more excited about SARS-CoV-5 (Captain Trips).
mrguyorama 3y ago
Generating bioweapons is not a trivial task even if you know exactly "how to do it". Ask any biologist how easy it is to grow X your very first attempt. Then remember buying starter cultures for most bioweapons isn't exactly easy and something anyone is allowed to do. Even an omniscient AI cannot overcome a skill issue