5 ms·

The problem is that we actually have lots of examples of the AI "secretly" being far smarter than it acts - prompt design. Tiny details in prompts can make huge

The problem is that we actually have lots of examples of the AI "secretly" being far smarter than it acts - prompt design. Tiny details in prompts can make huge differences in task performance.

LegionMammal978 4y ago
How is this at all supposed to indicate hidden intelligence? Does it not just indicate that current models are limited in that they don't associate some prompts with the desired task as well as they associate other prompts with the task?
- FeepingCreature 4y ago
  I'm not saying it indicates deliberately hidden intelligence, I'm just saying it's precedence for models already acting with less capability than they have.
- FeepingCreature 4y ago
  To clarify: what I'm saying is it may be unfalsifiable, but we have incontrovertible evidence that a similarly unfalsifiable thesis happens to be true. So we can't just reject it out of hand on that basis.
  - LegionMammal978 4y ago
    I just don't see how the two scenarios can be fairly compared. Prompt design, at least as far as I have seen, rarely allows a present-day model to perform a task that no one has ever seen it perform before; it can just allow it to perform far more consistently across different instances of the task. Contrast this with the proposed scenario, where a model can hold great deceptive capabilities, while outwardly never showing a deep understanding of its operators' mental states until it strikes. In my view, that would be like finding a secret prompt that lets GPT-3 emulate a fully sentient person with goals and desires. Sure, there's nothing physically ruling it out, but to me that kind of speculation will never be worthwhile until we have real evidence that AI models can possess (or emulate) the necessary theory of mind. Otherwise I'd have to spend my time worrying about any number of other Sufficiently Hidden Conspiracies that go far beyond the everyday conspiracies we know about.
    - FeepingCreature 4y ago
      I think we're less arguing about fundamentals and more about the precise bounds required. I think GPT-3 contained extremely surprising prompt-unlocked capabilities ("Let's go through it step by step." spawned a whole field of research) and also the AI doesn't need to never leak its deceptive skills, they just need to not be taken seriously. In other words, it's enough to cause danger if the AI is merely bad at deception given the "wrong" prompt.
      - LegionMammal978 4y ago
        Sure, that is a risk. I'm just saying that I'd rather wait until that point where we actually see AIs making poor attempts to estimate their operators' knowledge and deceive them; then, I'd have no problem worrying about it. But I suspect that point may not come for a long while, so until then it remains speculation.
        FeepingCreature 4y ago
        You may enjoy this after-action report of a person being attacked by a hostile AI, played by ChatGPT: https://www.lesswrong.com/posts/9kQFure4hdDmRBNdH/how-it-feels-to-have-your-mind-hacked-by-an-ai https://www.lesswrong.com/posts/9kQFure4hdDmRBNdH/how-it-fee... Now to be clear, this is possibly the easiest mark conceivable, and the poster played into his own demise at any available opportunity. But we should expect the first marks to be easy marks. People looking at videos of Hitler today don't understand why anyone would ever follow the funny man with the moustache. But probably his oratory skill simply does not record well? There's a thing with rallies and speeches where the speaker is reactive and feeding back with the mood of the crowd that makes it very hard to understand from the outside what happened. I think it's a mistake to look at this post, which was clearly a feedback loop where the poster was doing a significant amount of the cognitive work, and think "well this specific approach would not have worked on me." The approach that will happen to you will be personalized. But also: when people were experimenting with ChatGPT, a huge fraction of participants were trying to get the bot to access the open internet. Luckily this was technically impossible. But it shows that this sort of danger simply does not feel real to people - or at any rate, the urge to poke the thing overwhelms it. It will be easy to say "well GPT is still very bad at deceiving people, this attempt basically doesn't matter, we don't need to worry." When thinking of a target for AI deception, don't think of yourself, think of the most emotionally exploitable person out of a hundred. (If it's publically available, out of a million.) It's not exactly that when interacting with the public, the network gets to "pick its targets". That would assign intent where there is none. But if the network has a deceptive mode, its targets will direct it into that mode through mutual feedback.

The problem is that we actually have lots of examples of the AI "secretly" being far smarter than it acts - prompt design. Tiny details in prompts can make huge differences in task performance.

LegionMammal978 4y ago

How is this at all supposed to indicate hidden intelligence? Does it not just indicate that current models are limited in that they don't associate some prompts with the desired task as well as they associate other prompts with the task?

FeepingCreature 4y ago
I'm not saying it indicates deliberately hidden intelligence, I'm just saying it's precedence for models already acting with less capability than they have.
FeepingCreature 4y ago
To clarify: what I'm saying is it may be unfalsifiable, but we have incontrovertible evidence that a similarly unfalsifiable thesis happens to be true. So we can't just reject it out of hand on that basis.
- LegionMammal978 4y ago
  I just don't see how the two scenarios can be fairly compared. Prompt design, at least as far as I have seen, rarely allows a present-day model to perform a task that no one has ever seen it perform before; it can just allow it to perform far more consistently across different instances of the task. Contrast this with the proposed scenario, where a model can hold great deceptive capabilities, while outwardly never showing a deep understanding of its operators' mental states until it strikes. In my view, that would be like finding a secret prompt that lets GPT-3 emulate a fully sentient person with goals and desires. Sure, there's nothing physically ruling it out, but to me that kind of speculation will never be worthwhile until we have real evidence that AI models can possess (or emulate) the necessary theory of mind. Otherwise I'd have to spend my time worrying about any number of other Sufficiently Hidden Conspiracies that go far beyond the everyday conspiracies we know about.
  - FeepingCreature 4y ago
    I think we're less arguing about fundamentals and more about the precise bounds required. I think GPT-3 contained extremely surprising prompt-unlocked capabilities ("Let's go through it step by step." spawned a whole field of research) and also the AI doesn't need to never leak its deceptive skills, they just need to not be taken seriously. In other words, it's enough to cause danger if the AI is merely bad at deception given the "wrong" prompt.
    - LegionMammal978 4y ago
      Sure, that is a risk. I'm just saying that I'd rather wait until that point where we actually see AIs making poor attempts to estimate their operators' knowledge and deceive them; then, I'd have no problem worrying about it. But I suspect that point may not come for a long while, so until then it remains speculation.
      - FeepingCreature 4y ago
        You may enjoy this after-action report of a person being attacked by a hostile AI, played by ChatGPT: https://www.lesswrong.com/posts/9kQFure4hdDmRBNdH/how-it-feels-to-have-your-mind-hacked-by-an-ai https://www.lesswrong.com/posts/9kQFure4hdDmRBNdH/how-it-fee... Now to be clear, this is possibly the easiest mark conceivable, and the poster played into his own demise at any available opportunity. But we should expect the first marks to be easy marks. People looking at videos of Hitler today don't understand why anyone would ever follow the funny man with the moustache. But probably his oratory skill simply does not record well? There's a thing with rallies and speeches where the speaker is reactive and feeding back with the mood of the crowd that makes it very hard to understand from the outside what happened. I think it's a mistake to look at this post, which was clearly a feedback loop where the poster was doing a significant amount of the cognitive work, and think "well this specific approach would not have worked on me." The approach that will happen to you will be personalized. But also: when people were experimenting with ChatGPT, a huge fraction of participants were trying to get the bot to access the open internet. Luckily this was technically impossible. But it shows that this sort of danger simply does not feel real to people - or at any rate, the urge to poke the thing overwhelms it. It will be easy to say "well GPT is still very bad at deceiving people, this attempt basically doesn't matter, we don't need to worry." When thinking of a target for AI deception, don't think of yourself, think of the most emotionally exploitable person out of a hundred. (If it's publically available, out of a million.) It's not exactly that when interacting with the public, the network gets to "pick its targets". That would assign intent where there is none. But if the network has a deceptive mode, its targets will direct it into that mode through mutual feedback.