3 ms·

These are things you can just look up. The LLM contributes nothing that the internet didn't already provide. Or even textbooks before that. What I mean is actua

by ke88y 3y ago

These are things you can just look up. The LLM contributes nothing that the internet didn't already provide. Or even textbooks before that. What I mean is actually new capabilities, which weren't available to non-state actors with a broadband connection before. Also, sarin gas doesn't pose an existential risk to humanity. Or, in the sense that it could, the cat's out of the bag and I'm not sure why we're talking about LLMs as that seems like a dangerous distraction from the real problem, right? > I see nothing here that can't be replicated in principle by a not too large non-state actor. It's not like some people haven't already tried this when AI didn't exist in its modern form To my point, they didn't "try"!!! They DID. Without LLMs. Resulting in 13 deaths and thousands of injuries. I'm not sure that LLMs significantly increase the attack surface here. Possibly they do, and there are some mitigations we could introduce such that the barrier to using LLMs for this sort of thing are higher than the barrier to doing the bad thing in the first place without LLMs. But even in that case, it's not existential. And nowhere I have stated LLMs don't pose risks; they do. My issue is with AGI and x-risk prognostications.

yyyk 3y ago
>These are things you can just look up. The LLM contributes nothing that the internet didn't already provide The internet doesn't allow one to design _new_ weapons, possibly way more effective (which if you read carefully the first story, this one does). >I'm not sure that LLMs significantly increase the attack surface here. Being able to ask an AI to develop new deadly varieties which we'll not be able to detect or cure, and may be easy to produce, doesn't increase attack surface? >Also, sarin gas doesn't pose an existential risk to humanity Is x-risk the only thing we care about? The entire thread started with arguing x-risk is a distraction. I would be very slightly more comfortable with that argument if people took 'ordinary' risks seriously. As it is, all camps have their heads in the sand in different ways. The illusion here is that AI advancement changes nothing, so the only thing worth discussing are variations of the current culture war issues, when obviously it does change everything even if completely put aside AGI/alignment arguments. e.g. If labour won't matter for productivity that has very grim political implications. >To my point, they didn't "try"!!! They DID. That's the point. Give me an x-risk scenario the doomers warn about, and I'll find you a group of humans which very much want the exact scenario (or something essentially indistinguishable for 99% of humanity) to happen and will happily use AI if it helps them. Amusingly, alignment research is unlikely to help there - it can be argued to increase the risk from humans. >there are some mitigations we could introduce such that the barrier to using LLMs for this sort of thing There are many things we can do in theory to mitigate all sorts of issues, which have the nice property of never ever being done. e.g. Yud's favorite disaster scenario appears to be a custom built virus. This relies on biolabs accepting random orders, which leaves the question of why are we allowing this at all (AI or not)? There's no good reason for allowing most crypto to exist, given its current effects on society, even before AGI comes into question, yet we allow it for what reason exactly? If there's any risk here at all, we can safely rely on humanity doing nothing before anything happens - but in the case of x-risk actually existing, there's no reason to assume we'll have a second chance.
- ke88y 3y ago
  This entire category falls into my "numerical siulations at national labs" category of "don't care". If you wanted to use a bioweapon to kill a bunch of people, you would ignore the DeepCE paper and use weapons that have existed for decades. Existing weapons would be easier to design, easier to manufacture, easier to deploy, and more effective at killing. Computational drug discovery is not new, to put it mildly, and neither is the use of computation to design more effective weapons. Hell, the Harvard IBM Mark I was designed to help with the Manhattan project. There are huge barriers to entry between "know how to design/build/deploy a nuke/bioweapon" and "can actually do it". And that's how I feel about AI-for-weapons in general: the people who it helps can already make more effective weapons today if they want to. It's not the risk of using WMDs doesn't exist. It's that WMDs are already so deadly that our primary defense is just that there's a huge gap between "I know in principle how to design a nuke/bioweapon" and "I can actually design and deploy the weapon". I don't see how AI changes that equation. > Is x-risk the only thing we care about? The entire thread started with arguing x-risk is a distraction. I would be very slightly more comfortable with that argument if people took 'ordinary' risks seriously. Discussion of x-risk annoys me precisely because it's a distraction from working on real risks. > That's the point. Give me an x-risk scenario the doomers warn about, and I'll find you a group of humans which very much want the exact scenario (or something essentially indistinguishable for 99% of humanity) to happen and will happily use AI if it helps them. Amusingly, alignment research is unlikely to help there - it can be argued to increase the risk from humans. Right, but 1. those humans have existed for a long time, 2. public models don't provide them with a tool more or less powerful than the internet, and 3. to the extent that models like DeepCE help with discovery, someone with the knowledge and resources to actually operationalize this information wouldn't have needed DeepCE to do incredible amounts of damage. Again, I'm not saying there is no attack surface here. I'm saying that AI doesn't meaningfully change that landscape because the barrier to operationalizing is high enough that by the time you can operationalize it's unclear why you need to model -- that you couldn't have made the a similar discovery with a bit of extra time or even just used something off the shelf to the same effect. Or, to put it another way: killing a ton of people is shockingly easy in today's world. That is scary. But x-risk from superhuman AGI is a massive red herring, and even narrow AI for particular tasks such as drug discovery is honestly mostly unrelated to this observation. >>there are some mitigations we could introduce such that the barrier to using LLMs for this sort of thing >There are many things we can do in theory to mitigate all sorts of issues, which have the nice property of never ever being done. Speak for yourself. Mitigating real risks that could actually happen is what I work on every day. The people advocating for working on x-risk -- and the people working on x-risk -- are mostly writing sci-fi and doing philosophy of mind. At a minimum it's not useful. Anyways, at the very least, even if you want to prevent these x-risk scenarios, then focusing efforts on more concrete safety and controllability problems is probably the best path forward anyways.

by ke88y 3y ago

yyyk 3y ago

>These are things you can just look up. The LLM contributes nothing that the internet didn't already provide The internet doesn't allow one to design _new_ weapons, possibly way more effective (which if you read carefully the first story, this one does). >I'm not sure that LLMs significantly increase the attack surface here. Being able to ask an AI to develop new deadly varieties which we'll not be able to detect or cure, and may be easy to produce, doesn't increase attack surface? >Also, sarin gas doesn't pose an existential risk to humanity Is x-risk the only thing we care about? The entire thread started with arguing x-risk is a distraction. I would be very slightly more comfortable with that argument if people took 'ordinary' risks seriously. As it is, all camps have their heads in the sand in different ways. The illusion here is that AI advancement changes nothing, so the only thing worth discussing are variations of the current culture war issues, when obviously it does change everything even if completely put aside AGI/alignment arguments. e.g. If labour won't matter for productivity that has very grim political implications. >To my point, they didn't "try"!!! They DID. That's the point. Give me an x-risk scenario the doomers warn about, and I'll find you a group of humans which very much want the exact scenario (or something essentially indistinguishable for 99% of humanity) to happen and will happily use AI if it helps them. Amusingly, alignment research is unlikely to help there - it can be argued to increase the risk from humans. >there are some mitigations we could introduce such that the barrier to using LLMs for this sort of thing There are many things we can do in theory to mitigate all sorts of issues, which have the nice property of never ever being done. e.g. Yud's favorite disaster scenario appears to be a custom built virus. This relies on biolabs accepting random orders, which leaves the question of why are we allowing this at all (AI or not)? There's no good reason for allowing most crypto to exist, given its current effects on society, even before AGI comes into question, yet we allow it for what reason exactly? If there's any risk here at all, we can safely rely on humanity doing nothing before anything happens - but in the case of x-risk actually existing, there's no reason to assume we'll have a second chance.

ke88y 3y ago
This entire category falls into my "numerical siulations at national labs" category of "don't care". If you wanted to use a bioweapon to kill a bunch of people, you would ignore the DeepCE paper and use weapons that have existed for decades. Existing weapons would be easier to design, easier to manufacture, easier to deploy, and more effective at killing. Computational drug discovery is not new, to put it mildly, and neither is the use of computation to design more effective weapons. Hell, the Harvard IBM Mark I was designed to help with the Manhattan project. There are huge barriers to entry between "know how to design/build/deploy a nuke/bioweapon" and "can actually do it". And that's how I feel about AI-for-weapons in general: the people who it helps can already make more effective weapons today if they want to. It's not the risk of using WMDs doesn't exist. It's that WMDs are already so deadly that our primary defense is just that there's a huge gap between "I know in principle how to design a nuke/bioweapon" and "I can actually design and deploy the weapon". I don't see how AI changes that equation. > Is x-risk the only thing we care about? The entire thread started with arguing x-risk is a distraction. I would be very slightly more comfortable with that argument if people took 'ordinary' risks seriously. Discussion of x-risk annoys me precisely because it's a distraction from working on real risks. > That's the point. Give me an x-risk scenario the doomers warn about, and I'll find you a group of humans which very much want the exact scenario (or something essentially indistinguishable for 99% of humanity) to happen and will happily use AI if it helps them. Amusingly, alignment research is unlikely to help there - it can be argued to increase the risk from humans. Right, but 1. those humans have existed for a long time, 2. public models don't provide them with a tool more or less powerful than the internet, and 3. to the extent that models like DeepCE help with discovery, someone with the knowledge and resources to actually operationalize this information wouldn't have needed DeepCE to do incredible amounts of damage. Again, I'm not saying there is no attack surface here. I'm saying that AI doesn't meaningfully change that landscape because the barrier to operationalizing is high enough that by the time you can operationalize it's unclear why you need to model -- that you couldn't have made the a similar discovery with a bit of extra time or even just used something off the shelf to the same effect. Or, to put it another way: killing a ton of people is shockingly easy in today's world. That is scary. But x-risk from superhuman AGI is a massive red herring, and even narrow AI for particular tasks such as drug discovery is honestly mostly unrelated to this observation. >>there are some mitigations we could introduce such that the barrier to using LLMs for this sort of thing >There are many things we can do in theory to mitigate all sorts of issues, which have the nice property of never ever being done. Speak for yourself. Mitigating real risks that could actually happen is what I work on every day. The people advocating for working on x-risk -- and the people working on x-risk -- are mostly writing sci-fi and doing philosophy of mind. At a minimum it's not useful. Anyways, at the very least, even if you want to prevent these x-risk scenarios, then focusing efforts on more concrete safety and controllability problems is probably the best path forward anyways.