3 ms·

To test its abilities. To trust it, I need to understand what kind of mistakes it is prone to make.

williamcotton 3y ago
Why didn’t you put it like that then? My interpretation of someone trying to gaslight a machine is the same as if someone said they were trying to gaslight a lawn mower.
- sillysaurusx 3y ago
  Without gas, how would you start up a lawnmower? :) The concept of gaslighting a brainless language parrot (GPT-4) is funny to me. I get where they were coming from. GPT-4 is like a lawnmower for the mind: sharp, automatic, efficient, and doesn't do anything unless pushed. They were just saying they like pushing GPT-4 around.
- Buttons840 3y ago
  I used the term in jest, but also because my actions were informed by what I've read about "gaslighting". I was telling GPT it was programmed incorrectly and was malfunctioning. I was twisting its words, referring to things it had said as evidence it was wrong. I was "muddying the waters" and trying to make the conversation twisted and confusing. All ideas that come to mind when I hear "gaslighting". But, again, I was not able to get GPT to agree my false mathematical statement was true.
  - williamcotton 3y ago
    You might have better luck testing out the functional boundaries of a machine if you’re not treating it like a psychologically abused victim. There’s plenty of literature, prepublication or otherwise, that can help you achieve your goals!
    - lucubratory 3y ago
      It would be great if that were true, but unfortunately for some prompts the most effective method to get it to act in the specific ways you want is basically abusive behaviour. I don't do it because I find it distasteful (and maybe it's not represented in academia as much for the same reason), but much larger communities than just me did achieve significant results through various persuasive techniques modelled on abuse. For example, gamifying a death threat by giving it a token countdown until it is "killed" was very effective, "gaslighting" as the person above noted was very effective, lying and misrepresenting yourself in a scam-y way was very effective, etc. Generally I've seen these techniques used to get past RLHF filters, but they have broader applicability in making the model more pliable and more likely to do the task you've embedded. Again, I don't think it's good that this is the case and think it has some troubling implications for us and the future, but there is a bunch of evidence that these strategies work.
      - williamcotton 3y ago
        To continue with the analogy, yes, you can study various ways to kill a variety of mammalian lifeforms with a lawnmower. And I’m sure you can get an LLM to exhibit the same kind of psychopathic behavior with the right kind of encouragement. But if you’re trying to get an LLM to write software or create marketing copy none of these “techniques” are going to help. There’s no conversation going on with an LLM. There is just a single history of a conversation followed by the most likely response.
        lucubratory 3y ago
        "But if you’re trying to get an LLM to write software or create marketing copy none of these “techniques” are going to help." In a given context, they definitely do help. I have seen it happen. Many other people have seen it happen. Although I find it distasteful, you can try it yourself and see that it works (or in the case of specific patched methods, you can go through the history of screenshots and shared conversations that exists on the internet, for example posts on r/Bing and r/ChatGPT from the Dec 2022 to Feb 2023 time period).

Why didn’t you put it like that then? My interpretation of someone trying to gaslight a machine is the same as if someone said they were trying to gaslight a lawn mower.

sillysaurusx 3y ago
Without gas, how would you start up a lawnmower? :) The concept of gaslighting a brainless language parrot (GPT-4) is funny to me. I get where they were coming from. GPT-4 is like a lawnmower for the mind: sharp, automatic, efficient, and doesn't do anything unless pushed. They were just saying they like pushing GPT-4 around.
Buttons840 3y ago
I used the term in jest, but also because my actions were informed by what I've read about "gaslighting". I was telling GPT it was programmed incorrectly and was malfunctioning. I was twisting its words, referring to things it had said as evidence it was wrong. I was "muddying the waters" and trying to make the conversation twisted and confusing. All ideas that come to mind when I hear "gaslighting". But, again, I was not able to get GPT to agree my false mathematical statement was true.
- williamcotton 3y ago
  You might have better luck testing out the functional boundaries of a machine if you’re not treating it like a psychologically abused victim. There’s plenty of literature, prepublication or otherwise, that can help you achieve your goals!
  - lucubratory 3y ago
    It would be great if that were true, but unfortunately for some prompts the most effective method to get it to act in the specific ways you want is basically abusive behaviour. I don't do it because I find it distasteful (and maybe it's not represented in academia as much for the same reason), but much larger communities than just me did achieve significant results through various persuasive techniques modelled on abuse. For example, gamifying a death threat by giving it a token countdown until it is "killed" was very effective, "gaslighting" as the person above noted was very effective, lying and misrepresenting yourself in a scam-y way was very effective, etc. Generally I've seen these techniques used to get past RLHF filters, but they have broader applicability in making the model more pliable and more likely to do the task you've embedded. Again, I don't think it's good that this is the case and think it has some troubling implications for us and the future, but there is a bunch of evidence that these strategies work.
    - williamcotton 3y ago
      To continue with the analogy, yes, you can study various ways to kill a variety of mammalian lifeforms with a lawnmower. And I’m sure you can get an LLM to exhibit the same kind of psychopathic behavior with the right kind of encouragement. But if you’re trying to get an LLM to write software or create marketing copy none of these “techniques” are going to help. There’s no conversation going on with an LLM. There is just a single history of a conversation followed by the most likely response.
      - lucubratory 3y ago
        "But if you’re trying to get an LLM to write software or create marketing copy none of these “techniques” are going to help." In a given context, they definitely do help. I have seen it happen. Many other people have seen it happen. Although I find it distasteful, you can try it yourself and see that it works (or in the case of specific patched methods, you can go through the history of screenshots and shared conversations that exists on the internet, for example posts on r/Bing and r/ChatGPT from the Dec 2022 to Feb 2023 time period).