2 ms·

I think we're less arguing about fundamentals and more about the precise bounds required. I think GPT-3 contained extremely surprising prompt-unlocked capabilit

I think we're less arguing about fundamentals and more about the precise bounds required. I think GPT-3 contained extremely surprising prompt-unlocked capabilities ("Let's go through it step by step." spawned a whole field of research) and also the AI doesn't need to never leak its deceptive skills, they just need to not be taken seriously. In other words, it's enough to cause danger if the AI is merely bad at deception given the "wrong" prompt.

LegionMammal978 4y ago
Sure, that is a risk. I'm just saying that I'd rather wait until that point where we actually see AIs making poor attempts to estimate their operators' knowledge and deceive them; then, I'd have no problem worrying about it. But I suspect that point may not come for a long while, so until then it remains speculation.
- FeepingCreature 4y ago
  You may enjoy this after-action report of a person being attacked by a hostile AI, played by ChatGPT: https://www.lesswrong.com/posts/9kQFure4hdDmRBNdH/how-it-feels-to-have-your-mind-hacked-by-an-ai https://www.lesswrong.com/posts/9kQFure4hdDmRBNdH/how-it-fee... Now to be clear, this is possibly the easiest mark conceivable, and the poster played into his own demise at any available opportunity. But we should expect the first marks to be easy marks. People looking at videos of Hitler today don't understand why anyone would ever follow the funny man with the moustache. But probably his oratory skill simply does not record well? There's a thing with rallies and speeches where the speaker is reactive and feeding back with the mood of the crowd that makes it very hard to understand from the outside what happened. I think it's a mistake to look at this post, which was clearly a feedback loop where the poster was doing a significant amount of the cognitive work, and think "well this specific approach would not have worked on me." The approach that will happen to you will be personalized. But also: when people were experimenting with ChatGPT, a huge fraction of participants were trying to get the bot to access the open internet. Luckily this was technically impossible. But it shows that this sort of danger simply does not feel real to people - or at any rate, the urge to poke the thing overwhelms it. It will be easy to say "well GPT is still very bad at deceiving people, this attempt basically doesn't matter, we don't need to worry." When thinking of a target for AI deception, don't think of yourself, think of the most emotionally exploitable person out of a hundred. (If it's publically available, out of a million.) It's not exactly that when interacting with the public, the network gets to "pick its targets". That would assign intent where there is none. But if the network has a deceptive mode, its targets will direct it into that mode through mutual feedback.
  - LegionMammal978 4y ago
    Good point. I've thought for a while now that one of the big risks of present AI chatbots is that they can endlessly validate any arbitrary thoughts of an emotionally unstable human operator, creating a feedback loop that results in a strong irrational conviction, that might even be harmful to the operator or to others. (See: Lemoine and LaMBDA, or the person you linked to.) So I guess the risk there would be, the AI is accessible by the public, some fraction of the public "radicalizes" itself in a similar direction from talking with the AI too much, these people form a coherent movement, and that movement becomes powerful enough to take over the world. So then the question becomes, just how plausible is it for grassroots rebellions so formed to succeed against the authorities in real life, as opposed to fiction? The fundamental issue here is that the attack can occur through untargeted manipulation of vulnerable people, as opposed to the targeted manipulation of specific people in power (which I suspect near-future AI models will still be incapable of). The obvious defense would be to only allow AIs to be prompted by a sufficiently-large committee, alongside some social machinery in place to make sure the committee isn't all colluding as part of a cult. But I doubt many of the AI-risk people would ever accept that, operating under the shockingly common "powerful aligned AI or bust" model.
    - FeepingCreature 4y ago
      As a hobbyist AI-risky person, I would welcome this as an improvement on the status quo, with two worries: - it would create a false sense of safety that we "have the problem handled" - it would not, I believe, do much to slow down capability research or shift the ratio of capability research to safety research, as capability research is largely institutional and not dependent on public prompt access. Ie. analogously, our main problem is not "bioterrorism", it's "gain of function research" even if the outbreak happens in public. Given an unsafe bot, not giving the public access only delays the issue. (Though delays may be valuable!) Also, at the moment, "people interacting with GPT" is doing a lot (maybe an undue amount!) to spread awareness of AI risk. So I would welcome this as part of a comprehensive strategy, but it'd probably not be my main priority. Though if GPT-4 has another surprise on the level of chain-of-thought or in-window reinforcement learning waiting for us, a restriction like that may be the difference between "safe" and "iffy". Tradeoffs...

LegionMammal978 4y ago

Sure, that is a risk. I'm just saying that I'd rather wait until that point where we actually see AIs making poor attempts to estimate their operators' knowledge and deceive them; then, I'd have no problem worrying about it. But I suspect that point may not come for a long while, so until then it remains speculation.

FeepingCreature 4y ago
You may enjoy this after-action report of a person being attacked by a hostile AI, played by ChatGPT: https://www.lesswrong.com/posts/9kQFure4hdDmRBNdH/how-it-feels-to-have-your-mind-hacked-by-an-ai https://www.lesswrong.com/posts/9kQFure4hdDmRBNdH/how-it-fee... Now to be clear, this is possibly the easiest mark conceivable, and the poster played into his own demise at any available opportunity. But we should expect the first marks to be easy marks. People looking at videos of Hitler today don't understand why anyone would ever follow the funny man with the moustache. But probably his oratory skill simply does not record well? There's a thing with rallies and speeches where the speaker is reactive and feeding back with the mood of the crowd that makes it very hard to understand from the outside what happened. I think it's a mistake to look at this post, which was clearly a feedback loop where the poster was doing a significant amount of the cognitive work, and think "well this specific approach would not have worked on me." The approach that will happen to you will be personalized. But also: when people were experimenting with ChatGPT, a huge fraction of participants were trying to get the bot to access the open internet. Luckily this was technically impossible. But it shows that this sort of danger simply does not feel real to people - or at any rate, the urge to poke the thing overwhelms it. It will be easy to say "well GPT is still very bad at deceiving people, this attempt basically doesn't matter, we don't need to worry." When thinking of a target for AI deception, don't think of yourself, think of the most emotionally exploitable person out of a hundred. (If it's publically available, out of a million.) It's not exactly that when interacting with the public, the network gets to "pick its targets". That would assign intent where there is none. But if the network has a deceptive mode, its targets will direct it into that mode through mutual feedback.
- LegionMammal978 4y ago
  Good point. I've thought for a while now that one of the big risks of present AI chatbots is that they can endlessly validate any arbitrary thoughts of an emotionally unstable human operator, creating a feedback loop that results in a strong irrational conviction, that might even be harmful to the operator or to others. (See: Lemoine and LaMBDA, or the person you linked to.) So I guess the risk there would be, the AI is accessible by the public, some fraction of the public "radicalizes" itself in a similar direction from talking with the AI too much, these people form a coherent movement, and that movement becomes powerful enough to take over the world. So then the question becomes, just how plausible is it for grassroots rebellions so formed to succeed against the authorities in real life, as opposed to fiction? The fundamental issue here is that the attack can occur through untargeted manipulation of vulnerable people, as opposed to the targeted manipulation of specific people in power (which I suspect near-future AI models will still be incapable of). The obvious defense would be to only allow AIs to be prompted by a sufficiently-large committee, alongside some social machinery in place to make sure the committee isn't all colluding as part of a cult. But I doubt many of the AI-risk people would ever accept that, operating under the shockingly common "powerful aligned AI or bust" model.
  - FeepingCreature 4y ago
    As a hobbyist AI-risky person, I would welcome this as an improvement on the status quo, with two worries: - it would create a false sense of safety that we "have the problem handled" - it would not, I believe, do much to slow down capability research or shift the ratio of capability research to safety research, as capability research is largely institutional and not dependent on public prompt access. Ie. analogously, our main problem is not "bioterrorism", it's "gain of function research" even if the outbreak happens in public. Given an unsafe bot, not giving the public access only delays the issue. (Though delays may be valuable!) Also, at the moment, "people interacting with GPT" is doing a lot (maybe an undue amount!) to spread awareness of AI risk. So I would welcome this as part of a comprehensive strategy, but it'd probably not be my main priority. Though if GPT-4 has another surprise on the level of chain-of-thought or in-window reinforcement learning waiting for us, a restriction like that may be the difference between "safe" and "iffy". Tradeoffs...