3 ms·

The problem is that centralised control of AI won't defend against anything, because the persons with centralised control can still do whatever they like. Howe

The problem is that centralised control of AI won't defend against anything, because the persons with centralised control can still do whatever they like. However, bioweapons are not expensive or difficult even for human hobbyists. They are trivial. Some reasonably large fraction of physicians can easily create a bioweapon, almost all university professors in biomedicine related fields and many PhD students in biomedicine related fields. Bioweapons don't need LLMs. They are trivial. The only reason people don't build them is that they don't want people dead. If you want to do ethnic targeting or something like that you of course need some more effort-- to do actual research in medicine, but if you don't and just want a pandemic, or to kill a couple of million people in a certain city or region, that's perfectly feasible. Bioweapons creation isn't some kind 'Oh, we're so smart, look how dangerous we are', it's trivial and anyone who knows anything can do it. Something so stupid and so easy you can't publish it, not because it's dangerous, but because it's of no scientific interest and not novel. So limiting LLMs out of fear of this is a bunch of silliness. The only thing to do is be nice to your physicians and biomedicine researchers and not make them hate the world.

hackerlight 3y ago
Don't think about it in terms of the binary lens of easy vs hard. There are probability frequencies and gradations to outcomes. What percentage of the general population can create a respiratory virus with Covid's infectiousness and Ebola's lethality? Close to 0%. Maybe 0% if you're talking about a single human that isn't operating with the support of a team of people and equipment in a lab. So effectively 0% because an entire lab probably isn't going to do such a stupid thing, that's the domain of lone terrorists. What does that % go up to if everyone is equipped with multiple instances of autonomous AGI? We don't know. It could remain at 0% (due to real world constraints like lack of access to required equipment) or it could go up to 10% or 100%. The point is that the probability and expected magnitude of asymmetric offensive risks goes up, probably significantly. If 10% of the world can suddenly do it, that's 800 million people. For it to not go wrong, all 800 million have to individually decide to not engage in the behavior, which is a probability of (1-p)^800000000. Even if p is very small (which it is, because the large majority of people aren't insane), that number goes to zero quickly. This is the point of the asymmetry observation, you just need 1 insane person to make things bad if each person is equipped with very powerful technology, whereas in the old world large groups of people need to jointly decide to be bad which is much less likely to happen.
- impossiblefork 3y ago
  I don't agree. Fiddling with viruses in the way you describe can be done easily by individual researchers and PhD students. Probably many tens of thousands of people can do this. But of course it's close to 0%, but the number of people who can cook béarnaise sauce is also close to 0%. I don't think it'll go up much. Most of it is physical finesse in dealing with yeast and bacterial cultures, knowing how to cook the growth medium, knowing how to kill the wrong bacteria, knowing how to debug your procedures, etc. If they can just order plasmids etc., I think the easiest methods are actually accessible to skilled highschoolers, like the kind of people who in highschool train to be laboratory technicians. There's no way to protect oneself from any of this. One just has to accept that the biomedicine people can kill everybody and it's going to be easy for them to do this, and that it's going to be easy for them to do this forever.
  - bbor 3y ago
    There's no way to protect oneself from any of this. One just has to accept that the biomedicine people can kill everybody and it's going to be easy for them to do this, and that it's going to be easy for them to do this forever. With absolutely no offense intended: this reads like dialogue from the well-meaning second-act villain in a comic book movie. You’re obviously knowledgeable about the specifics, so I’m not saying you’re wrong. I for one am optimistic we can keep biomedical safety at or near current levels
    - impossiblefork 3y ago
      Maybe I'm knowledgeable, but I obviously haven't tried these things that I'm fairly sure are relatively easy, but I can't absolutely say that it's as easy as I have said. It can be fiddly to figure out experimental procedures, and debugging is hard, so it's possible that this really only is something that PhD students and above can do, but I think it's accessible to skilled to hobbyists. But I should say my biomedicine knowledge consists of a special highschool biotechnology program, nothing more. Basically we put plasmids into bacteria and grew them, maybe we did something with yeast, we did DNA amplification and other easy experiments of that sort. This is the extent of my biomedicine knowledge. I think biomedicine people just don't want to kill everybody. The idea of biomedical safety at our current levels, that's what I object to: there isn't any. The safety consists entirely in the biomedicine people being decent. They are self-interested and intelligent, so a society genuinely shits on them, then the risk increases. After all, this stuff is really very easy. But as long as one has democracy, reasonable order, etc., then the fact that they're decent means that there is very little risk. It's the same thing with political assassinations. The politicians aren't alive because of clever guards and security arrangements, they are quite vulnerable in fact, but most people want democracy, civilisation, etc. are willing to accept even bad politicians that they oppose if that's what everybody else wants. But if you think that you're safe because of your guards and security arrangements, then you get the wrong attitude.

hackerlight 3y ago

Don't think about it in terms of the binary lens of easy vs hard. There are probability frequencies and gradations to outcomes. What percentage of the general population can create a respiratory virus with Covid's infectiousness and Ebola's lethality? Close to 0%. Maybe 0% if you're talking about a single human that isn't operating with the support of a team of people and equipment in a lab. So effectively 0% because an entire lab probably isn't going to do such a stupid thing, that's the domain of lone terrorists. What does that % go up to if everyone is equipped with multiple instances of autonomous AGI? We don't know. It could remain at 0% (due to real world constraints like lack of access to required equipment) or it could go up to 10% or 100%. The point is that the probability and expected magnitude of asymmetric offensive risks goes up, probably significantly. If 10% of the world can suddenly do it, that's 800 million people. For it to not go wrong, all 800 million have to individually decide to not engage in the behavior, which is a probability of (1-p)^800000000. Even if p is very small (which it is, because the large majority of people aren't insane), that number goes to zero quickly. This is the point of the asymmetry observation, you just need 1 insane person to make things bad if each person is equipped with very powerful technology, whereas in the old world large groups of people need to jointly decide to be bad which is much less likely to happen.

impossiblefork 3y ago
I don't agree. Fiddling with viruses in the way you describe can be done easily by individual researchers and PhD students. Probably many tens of thousands of people can do this. But of course it's close to 0%, but the number of people who can cook béarnaise sauce is also close to 0%. I don't think it'll go up much. Most of it is physical finesse in dealing with yeast and bacterial cultures, knowing how to cook the growth medium, knowing how to kill the wrong bacteria, knowing how to debug your procedures, etc. If they can just order plasmids etc., I think the easiest methods are actually accessible to skilled highschoolers, like the kind of people who in highschool train to be laboratory technicians. There's no way to protect oneself from any of this. One just has to accept that the biomedicine people can kill everybody and it's going to be easy for them to do this, and that it's going to be easy for them to do this forever.
- bbor 3y ago
  There's no way to protect oneself from any of this. One just has to accept that the biomedicine people can kill everybody and it's going to be easy for them to do this, and that it's going to be easy for them to do this forever. With absolutely no offense intended: this reads like dialogue from the well-meaning second-act villain in a comic book movie. You’re obviously knowledgeable about the specifics, so I’m not saying you’re wrong. I for one am optimistic we can keep biomedical safety at or near current levels
  - impossiblefork 3y ago
    Maybe I'm knowledgeable, but I obviously haven't tried these things that I'm fairly sure are relatively easy, but I can't absolutely say that it's as easy as I have said. It can be fiddly to figure out experimental procedures, and debugging is hard, so it's possible that this really only is something that PhD students and above can do, but I think it's accessible to skilled to hobbyists. But I should say my biomedicine knowledge consists of a special highschool biotechnology program, nothing more. Basically we put plasmids into bacteria and grew them, maybe we did something with yeast, we did DNA amplification and other easy experiments of that sort. This is the extent of my biomedicine knowledge. I think biomedicine people just don't want to kill everybody. The idea of biomedical safety at our current levels, that's what I object to: there isn't any. The safety consists entirely in the biomedicine people being decent. They are self-interested and intelligent, so a society genuinely shits on them, then the risk increases. After all, this stuff is really very easy. But as long as one has democracy, reasonable order, etc., then the fact that they're decent means that there is very little risk. It's the same thing with political assassinations. The politicians aren't alive because of clever guards and security arrangements, they are quite vulnerable in fact, but most people want democracy, civilisation, etc. are willing to accept even bad politicians that they oppose if that's what everybody else wants. But if you think that you're safe because of your guards and security arrangements, then you get the wrong attitude.