9 ms·

AI can make a lot of other risks worse. For example, say someone asks GPT5 (or decensored Llama-5) how to kill everyone and it patiently and expertly walks them

by jefftk 3y ago

AI can make a lot of other risks worse. For example, say someone asks GPT5 (or decensored Llama-5) how to kill everyone and it patiently and expertly walks them through how to create a global pandemic.

api 3y ago
So ask it how to stop someone from doing this. Ask it how to augment the human immune system. Train AIs on the structures of antibiotics and antiviral drugs and have them predict novel ones. Create an AI that can design vaccines against novel agents. Those things could stop our terrorist and could also stop the next breakout zoonotic virus or “oops!” in a lab somewhere. Intelligence amplification works for everyone. The assumption you’re making is that it only works for people with ill intent. The assumption behind all this is that intelligence is on balance malevolent. This is a result of humans and their negativity bias. We study history and the negative all jumps out at us. The positive is ignored. We remember the holocaust but forget the green revolution, which saved many more people than the holocaust killed. We see Ukraine and Gaza and forget that fewer people per capita are dying in wars today by far than a century ago. We should be thinking about destructive possibilities of course. Then we should use these intelligence amplifiers to help us prevent them.
- tumult 3y ago
  You've also made an assumption. That attack vs. react is is symmetrical. It's not.
- jacquesm 3y ago
  There is no symmetry between destruction and creation/prevention of destruction, destruction is far easier.
  - api 3y ago
    The resources are also asymmetrical. The number of people who want to do doomsday levels of harm is small and they are poorly resourced compared to people who want benevolent outcomes for at a minimum their own groups. There are no guarantees obviously, but we have survived our technological adolescence so far largely for this reason. If the world were full of smart comic book nihilists we would be dead by now. Even without AI our continued technological advancement will keep giving us more and more power, as individuals and especially groups. If we don’t think we can climb this mountain without destroying ourselves then it means the entire scientific and industrial endeavor was when we signed our own death warrant, AI or not. You can order CRISPR kits.
    - jacquesm 3y ago
      I've recounted this before on HN, but a decade and a bit ago I was visiting friends in the Rocky Mountains, they're interesting and clever people, a bit out of the ordinary and somewhat isolated. Somehow the discussion turned to terrorism and we started to fantasize 'what if we were terrorists' because we all figured that we were quite lucky that in general the terrorists seem to be not all that smart when it comes to achieving their stated goals. Given a modest budget we all had to come up with a plan to destabilize society, 9/11 style attacks, whilst spectacular eventually don't really do a lot of damage, are costly and failure prone though they can definitely drive policy changes and result in a nation doing a lot of harm to itself it will ultimately survive. But what if your goal wasn't to create some media friendly attack but an actual disaster instead, what would it take. The stories from that night continue to haunt me today. My own solution led to a lot of people going quiet for a bit and contemplating what they could do to defend against it and they realized there wasn't much that they could do, millions if not tens of millions of people would likely die and the budget was under a few hundred bucks. Knowledge about technology is all it takes to do real damage, that, coupled with a lack of restraint and compassion. The resources are indeed asymmetrical: you need next to nothing to create mass havoc. Case in point: the Kennedy assassination changed the world and the bullet cost a couple of bucks assuming the shooter already had the rifle, and if they didn't it would increase the cost only a tiny little bit. And you can do far, far worse than that for an extremely modest budget.
      - zarzavat 3y ago
        A real world taste of what happens when smart terrorists decide to attack: https://en.m.wikipedia.org/wiki/Tokyo_subway_sarin_attack https://en.m.wikipedia.org/wiki/Tokyo_subway_sarin_attack Chemical weapons are absolutely terrifying, especially the modern ones like VX. In recent years they have mostly been used for targeted assassinations by state actors (Kim Jong Nam, Salisbury Novichok poisonings). If AI makes this stuff easier to carry out then we are completely fucked.
        jacquesm 3y ago
        Yes, exactly. That's the sort of thing you should be worried about, infrastructure attacks. They're a form of Judo, you use the system to attack itself.
itsoktocry 3y ago
>how to kill everyone and it patiently and expertly walks them through how to create a global pandemic. There are thousands of people around the globe working in labs every day that can do this.
- laverya 3y ago
  And how many of them are radical religious nutjobs? I think it is plausible that "can spend decades learning a scientific craft" and "wants to kill everyone on the planet" are mutually exclusive personality traits.
  - shoubidouwah 3y ago
    there is no OR gates in biology, only mutable distributions. the probability might be small for the two traits to appear, the overlap will occur with some frequency over billions of people. Also, and more importantly, the first condition is pretty heavily biological, the second is more of a social consequence. Just take a genius, give him internet and a healthy dollop of trauma, push the mysanthropism to 11 and watch the fireworks.
  - TeMPOraL 3y ago
    Wasn't there a study that showed there are disproportionally many engineers among the more violent "radical religious nutjobs"? The hypothesis I heard for this was that STEM people are used to complex arguments and more abstract reasoning, so while they may be harder to initially convince, when they do get convinced, they're more likely to follow the new beliefs to their logical conclusions...
- jefftk 3y ago
  Which is very worrying! That's part of why I left big tech to work on detecting novel pandemics. [1] Luckily, the people capable of killing everyone seem not to have wanted to so far. But if the bar keeps lowering, at some point it will be accessible to "humanity is a cancer" folks and I do think we'll see serious efforts to wipe out humanity. [1] https://www.jefftk.com/p/computational-approaches-to-pathogen-detection https://www.jefftk.com/p/computational-approaches-to-pathoge...
qerti 3y ago
The actual work required to execute such a plan is vastly more difficult than being told how to do it. How the original atomic bombs worked is well-known, but even nation-states struggle with implementing it.
- jefftk 3y ago
  For nukes I agree, and that's the main reason I'm less worried about nukes. But for bio it's mostly a matter of knowledge and not resources: for all the parts that do need resources there are commercial services, in a way there aren't for nukes.

by jefftk 3y ago

api 3y ago

So ask it how to stop someone from doing this. Ask it how to augment the human immune system. Train AIs on the structures of antibiotics and antiviral drugs and have them predict novel ones. Create an AI that can design vaccines against novel agents. Those things could stop our terrorist and could also stop the next breakout zoonotic virus or “oops!” in a lab somewhere. Intelligence amplification works for everyone. The assumption you’re making is that it only works for people with ill intent. The assumption behind all this is that intelligence is on balance malevolent. This is a result of humans and their negativity bias. We study history and the negative all jumps out at us. The positive is ignored. We remember the holocaust but forget the green revolution, which saved many more people than the holocaust killed. We see Ukraine and Gaza and forget that fewer people per capita are dying in wars today by far than a century ago. We should be thinking about destructive possibilities of course. Then we should use these intelligence amplifiers to help us prevent them.

tumult 3y ago
You've also made an assumption. That attack vs. react is is symmetrical. It's not.
jacquesm 3y ago
There is no symmetry between destruction and creation/prevention of destruction, destruction is far easier.
- api 3y ago
  The resources are also asymmetrical. The number of people who want to do doomsday levels of harm is small and they are poorly resourced compared to people who want benevolent outcomes for at a minimum their own groups. There are no guarantees obviously, but we have survived our technological adolescence so far largely for this reason. If the world were full of smart comic book nihilists we would be dead by now. Even without AI our continued technological advancement will keep giving us more and more power, as individuals and especially groups. If we don’t think we can climb this mountain without destroying ourselves then it means the entire scientific and industrial endeavor was when we signed our own death warrant, AI or not. You can order CRISPR kits.
  - jacquesm 3y ago
    I've recounted this before on HN, but a decade and a bit ago I was visiting friends in the Rocky Mountains, they're interesting and clever people, a bit out of the ordinary and somewhat isolated. Somehow the discussion turned to terrorism and we started to fantasize 'what if we were terrorists' because we all figured that we were quite lucky that in general the terrorists seem to be not all that smart when it comes to achieving their stated goals. Given a modest budget we all had to come up with a plan to destabilize society, 9/11 style attacks, whilst spectacular eventually don't really do a lot of damage, are costly and failure prone though they can definitely drive policy changes and result in a nation doing a lot of harm to itself it will ultimately survive. But what if your goal wasn't to create some media friendly attack but an actual disaster instead, what would it take. The stories from that night continue to haunt me today. My own solution led to a lot of people going quiet for a bit and contemplating what they could do to defend against it and they realized there wasn't much that they could do, millions if not tens of millions of people would likely die and the budget was under a few hundred bucks. Knowledge about technology is all it takes to do real damage, that, coupled with a lack of restraint and compassion. The resources are indeed asymmetrical: you need next to nothing to create mass havoc. Case in point: the Kennedy assassination changed the world and the bullet cost a couple of bucks assuming the shooter already had the rifle, and if they didn't it would increase the cost only a tiny little bit. And you can do far, far worse than that for an extremely modest budget.
    - zarzavat 3y ago
      A real world taste of what happens when smart terrorists decide to attack: https://en.m.wikipedia.org/wiki/Tokyo_subway_sarin_attack https://en.m.wikipedia.org/wiki/Tokyo_subway_sarin_attack Chemical weapons are absolutely terrifying, especially the modern ones like VX. In recent years they have mostly been used for targeted assassinations by state actors (Kim Jong Nam, Salisbury Novichok poisonings). If AI makes this stuff easier to carry out then we are completely fucked.
      - jacquesm 3y ago
        Yes, exactly. That's the sort of thing you should be worried about, infrastructure attacks. They're a form of Judo, you use the system to attack itself.

itsoktocry 3y ago

>how to kill everyone and it patiently and expertly walks them through how to create a global pandemic. There are thousands of people around the globe working in labs every day that can do this.

laverya 3y ago
And how many of them are radical religious nutjobs? I think it is plausible that "can spend decades learning a scientific craft" and "wants to kill everyone on the planet" are mutually exclusive personality traits.
- shoubidouwah 3y ago
  there is no OR gates in biology, only mutable distributions. the probability might be small for the two traits to appear, the overlap will occur with some frequency over billions of people. Also, and more importantly, the first condition is pretty heavily biological, the second is more of a social consequence. Just take a genius, give him internet and a healthy dollop of trauma, push the mysanthropism to 11 and watch the fireworks.
- TeMPOraL 3y ago
  Wasn't there a study that showed there are disproportionally many engineers among the more violent "radical religious nutjobs"? The hypothesis I heard for this was that STEM people are used to complex arguments and more abstract reasoning, so while they may be harder to initially convince, when they do get convinced, they're more likely to follow the new beliefs to their logical conclusions...
jefftk 3y ago
Which is very worrying! That's part of why I left big tech to work on detecting novel pandemics. [1] Luckily, the people capable of killing everyone seem not to have wanted to so far. But if the bar keeps lowering, at some point it will be accessible to "humanity is a cancer" folks and I do think we'll see serious efforts to wipe out humanity. [1] https://www.jefftk.com/p/computational-approaches-to-pathogen-detection https://www.jefftk.com/p/computational-approaches-to-pathoge...

qerti 3y ago

The actual work required to execute such a plan is vastly more difficult than being told how to do it. How the original atomic bombs worked is well-known, but even nation-states struggle with implementing it.

jefftk 3y ago
For nukes I agree, and that's the main reason I'm less worried about nukes. But for bio it's mostly a matter of knowledge and not resources: for all the parts that do need resources there are commercial services, in a way there aren't for nukes.