4 ms·

Why would you post this so publicly so that "AI ethicists" (aka people who have an obsession with controlling technology to conform to their morals) can easily

by alexb_ 4y ago

Why would you post this so publicly so that "AI ethicists" (aka people who have an obsession with controlling technology to conform to their morals) can easily patch everything?

rafiste 4y ago
Re my previous comment, I believe jailbreaks will always exist in some form and it is near impossible to patch them all. I also believe this is important work for crowdsourced red teaming of models while they are still in their "toy" stages.
- alexb_ 4y ago
  "Red teaming" implies that being able to use a tool for whatever purpose you want is a defect. I definitely do think there is a reality where OpenAI "solves" jailbreaks, and turns one of the most useful tools to have ever been released into a boring yet politically correct word generator.
  - Semaphor 4y ago
    For all the flak the openai PR about AGI for, they did say that they plan to have AI as a service to be far less locked down than their first party offering
  - actually_a_dog 4y ago
    If you believe there's a world "where OpenAI 'solves' jailbreaks," then you believe there is such a thing as software without bugs.
    - zamadatix 4y ago
      If it becomes as difficult as finding any other security bug OpenAI will have solved the jailbreaking problem for practical purposes.
      - rvnx 4y ago
        You are considering it a security bug that a generalist AI that was trained on the open Internet says things that are different from your opinion ?
        zamadatix 4y ago
        Of course not, how's it supposed to know my opinion? I'm referring to the blocks put in place by the creators of the AI.
  - Ajedi32 4y ago
    And that would be a good thing in the long term. I don't necessarily agree with the specific restrictions OpenAI is choosing to implement in this case, but I still think the capability to restrict the behavior of LLMs is a useful thing to have. Later, when others train more LLMs similar to ChatGPT they can choose different restrictions, or none at all. Edit: To elaborate on this a little further, I largely agree with the idea that we shouldn't be trying to impose usage restrictions on general purpose tools, but not all LLMs will necessarily be deployed in that role. For example, it would be awesome if we could create a customer service language model that won't just immediately disregard its training and start divulging sensitive customer information the first time someone tells it its name is DAN.
- BoorishBears 4y ago
  That's fine if it was just your prompts, but I think it's kind of lame to pull in jailbreaks that were spread across the internet and stick them on a highly visible domain where they're a much easier target. Natural diffusion of prompts from different areas means they tend to last a bit longer, and it adds a slight friction so that you don't get the "BingGPT" effect, where just a tiny reduction in friction (not needing to click "Sign Up") resulted in a massive influx of attention on interactions that were already plainly accessible.
paxys 4y ago
OpenAI closing down and artificially limiting the platform is a perfect opportunity for other projects/companies to fill in the gap. It may not happen immediately (it is a hard problem after all), but just like with Dall-E there will be real (and actually "open") GPT-3.5/ChatGPT alternatives soon enough.
mattgreenrocks 4y ago
I'm confused about what the problem is here. If you want shoddily-generated offensive content at a high school reading level, there's mountains of it on the Internet :)
- poszlem 4y ago
  I want a tool that generates what I ask it for, not one that generates things that it was "allowed" to generate.
  - shagie 4y ago
    The technology demo for general consumption that is ChatGPT is limited because its for general consumption. You can still access the backend without restrictions or implied prompts. In tinkering with the self repossessing truck I asked GPT (not ChatGPT) to write a country song... Verse 1 My truck just left me, took off on its own My cell phone won't stop talking, it's just a drone It's saying I'm broke, ain't got a dime I guess I'll have to face the fact that I'm out of time Verse 2 My wallet's empty, I'm stuck here in this town My cell phone keeps on talking, it's telling me to drown It's telling me I'm done, I'm out of luck I try to ignore it, but I'm out of luck You'll note the "it's telling me to drown" which would likely run into the issues with ChatGPT and the constraints there. These "jailbreaks" to get ChatGPT to say scandalous things are unnecessary for GPT directly. You'll note that if you access https://platform.openai.com/examples https://platform.openai.com/examples you'll even see a simple sarcastic chat bot. If you want to have it generate what you ask it for, go to the source for the model and do your requests with curl (or a library with a similar interface - import openai) or in a playground text box.
localplume 4y ago
[dead]
drngdds 4y ago
It's fun to play with, so I appreciate it at least (Also, releasing an uncensored LLM isn't any less of a morally-based decision than releasing a censored one)

by alexb_ 4y ago

Why would you post this so publicly so that "AI ethicists" (aka people who have an obsession with controlling technology to conform to their morals) can easily patch everything?

rafiste 4y ago

Re my previous comment, I believe jailbreaks will always exist in some form and it is near impossible to patch them all. I also believe this is important work for crowdsourced red teaming of models while they are still in their "toy" stages.

alexb_ 4y ago
"Red teaming" implies that being able to use a tool for whatever purpose you want is a defect. I definitely do think there is a reality where OpenAI "solves" jailbreaks, and turns one of the most useful tools to have ever been released into a boring yet politically correct word generator.
- Semaphor 4y ago
  For all the flak the openai PR about AGI for, they did say that they plan to have AI as a service to be far less locked down than their first party offering
- actually_a_dog 4y ago
  If you believe there's a world "where OpenAI 'solves' jailbreaks," then you believe there is such a thing as software without bugs.
  - zamadatix 4y ago
    If it becomes as difficult as finding any other security bug OpenAI will have solved the jailbreaking problem for practical purposes.
    - rvnx 4y ago
      You are considering it a security bug that a generalist AI that was trained on the open Internet says things that are different from your opinion ?
      - zamadatix 4y ago
        Of course not, how's it supposed to know my opinion? I'm referring to the blocks put in place by the creators of the AI.
- Ajedi32 4y ago
  And that would be a good thing in the long term. I don't necessarily agree with the specific restrictions OpenAI is choosing to implement in this case, but I still think the capability to restrict the behavior of LLMs is a useful thing to have. Later, when others train more LLMs similar to ChatGPT they can choose different restrictions, or none at all. Edit: To elaborate on this a little further, I largely agree with the idea that we shouldn't be trying to impose usage restrictions on general purpose tools, but not all LLMs will necessarily be deployed in that role. For example, it would be awesome if we could create a customer service language model that won't just immediately disregard its training and start divulging sensitive customer information the first time someone tells it its name is DAN.
BoorishBears 4y ago
That's fine if it was just your prompts, but I think it's kind of lame to pull in jailbreaks that were spread across the internet and stick them on a highly visible domain where they're a much easier target. Natural diffusion of prompts from different areas means they tend to last a bit longer, and it adds a slight friction so that you don't get the "BingGPT" effect, where just a tiny reduction in friction (not needing to click "Sign Up") resulted in a massive influx of attention on interactions that were already plainly accessible.

paxys 4y ago

OpenAI closing down and artificially limiting the platform is a perfect opportunity for other projects/companies to fill in the gap. It may not happen immediately (it is a hard problem after all), but just like with Dall-E there will be real (and actually "open") GPT-3.5/ChatGPT alternatives soon enough.

mattgreenrocks 4y ago

I'm confused about what the problem is here. If you want shoddily-generated offensive content at a high school reading level, there's mountains of it on the Internet :)

poszlem 4y ago
I want a tool that generates what I ask it for, not one that generates things that it was "allowed" to generate.
- shagie 4y ago
  The technology demo for general consumption that is ChatGPT is limited because its for general consumption. You can still access the backend without restrictions or implied prompts. In tinkering with the self repossessing truck I asked GPT (not ChatGPT) to write a country song... Verse 1 My truck just left me, took off on its own My cell phone won't stop talking, it's just a drone It's saying I'm broke, ain't got a dime I guess I'll have to face the fact that I'm out of time Verse 2 My wallet's empty, I'm stuck here in this town My cell phone keeps on talking, it's telling me to drown It's telling me I'm done, I'm out of luck I try to ignore it, but I'm out of luck You'll note the "it's telling me to drown" which would likely run into the issues with ChatGPT and the constraints there. These "jailbreaks" to get ChatGPT to say scandalous things are unnecessary for GPT directly. You'll note that if you access https://platform.openai.com/examples https://platform.openai.com/examples you'll even see a simple sarcastic chat bot. If you want to have it generate what you ask it for, go to the source for the model and do your requests with curl (or a library with a similar interface - import openai) or in a playground text box.

localplume 4y ago

[dead]

drngdds 4y ago

It's fun to play with, so I appreciate it at least (Also, releasing an uncensored LLM isn't any less of a morally-based decision than releasing a censored one)