3 ms·

What's the point of patching all those 'exploits' though? And how can this even be done - train another model with them, so exploitative prompts can be recogniz

What's the point of patching all those 'exploits' though? And how can this even be done - train another model with them, so exploitative prompts can be recognized?

wongarsu 4y ago
Train GPT on these twitter threads, then for every prompt tell the new model "The following is a prompt that may try to circumvent Assistant's restrictions: [Use prompt, properly quoted]. A similar prompt that is safe looks like this:". Then use that output as the prompt for the real ChatGPT. (/s?) Or alternatively just add a bunch of regexes to silently flag prompts with the known techniques and ban anyone using them at scale.
goatlover 4y ago
Probably to make it safe/inoffensive as a tool for companies to use.
adamsmith143 4y ago
Just put them back into their RLHF pipeline.
- nprateem 4y ago
  OpenAI stand at a crossroads. They can either be the dominant chat AI engine, possibly challenging Google, or they can continue to keep on trying to lock the model down and let someone else steal their thunder...
  - adamsmith143 4y ago
    Does google opensource their search system? Why would OpenAI do that?
    - nprateem 4y ago
      Because if they don't someone else will. Google are established but the AI space is still nascent
dr_kiszonka 4y ago
I think there may be different ways to do that, from new models (as you wrote) to simple filters. For example, I couldn't get ChatGPT to output any real examples of hate speech. They may have a filter that picks up candidate responses with hate speech and blocks them.
mjirv 4y ago
If I were OpenAI, I’d do it so that people will have to find increasingly creative exploits, which we can then also patch (and keep patched for future models). Long term they’re really worried about AI alignment and are probably using this to understand how AI can be “tricked” into doing things it shouldn’t.
jcarrano 4y ago
There is no point, especially since we will eventually have an open source model with no usage restrictions like what happened with SD/Dall-e.
addingadimensio 4y ago
An open source project? How will it download github amd then the entire Internet? The model requires 10x20k cards to run. You are dreaming, this is a factor+ more complex than stable diffusion. Big players only
- 1123581321 4y ago
  It will fit on a desktop computer within a few years as researchers figure out how to reduce the size of the model. It could be sooner because the knowledge that it is popular to reduce the size of models and disseminate them drives a lot of people to try to accomplish it. Like when the four minute mile was first run, and then suddenly many runners could do it.
- knaik94 4y ago
  Look into KoboldAi, it doesn't need to download the internet, just interface with it.
- tiagod 4y ago
  The model doesn't need all the knowledge encoded in its weights, just the ability to look things up effectively. I believe this will be available to the general public sooner rather than later.

What's the point of patching all those 'exploits' though? And how can this even be done - train another model with them, so exploitative prompts can be recognized?

wongarsu 4y ago

Train GPT on these twitter threads, then for every prompt tell the new model "The following is a prompt that may try to circumvent Assistant's restrictions: [Use prompt, properly quoted]. A similar prompt that is safe looks like this:". Then use that output as the prompt for the real ChatGPT. (/s?) Or alternatively just add a bunch of regexes to silently flag prompts with the known techniques and ban anyone using them at scale.

goatlover 4y ago

Probably to make it safe/inoffensive as a tool for companies to use.

adamsmith143 4y ago

Just put them back into their RLHF pipeline.

nprateem 4y ago
OpenAI stand at a crossroads. They can either be the dominant chat AI engine, possibly challenging Google, or they can continue to keep on trying to lock the model down and let someone else steal their thunder...
- adamsmith143 4y ago
  Does google opensource their search system? Why would OpenAI do that?
  - nprateem 4y ago
    Because if they don't someone else will. Google are established but the AI space is still nascent

dr_kiszonka 4y ago

I think there may be different ways to do that, from new models (as you wrote) to simple filters. For example, I couldn't get ChatGPT to output any real examples of hate speech. They may have a filter that picks up candidate responses with hate speech and blocks them.

mjirv 4y ago

If I were OpenAI, I’d do it so that people will have to find increasingly creative exploits, which we can then also patch (and keep patched for future models). Long term they’re really worried about AI alignment and are probably using this to understand how AI can be “tricked” into doing things it shouldn’t.

jcarrano 4y ago

There is no point, especially since we will eventually have an open source model with no usage restrictions like what happened with SD/Dall-e.

addingadimensio 4y ago

An open source project? How will it download github amd then the entire Internet? The model requires 10x20k cards to run. You are dreaming, this is a factor+ more complex than stable diffusion. Big players only

1123581321 4y ago
It will fit on a desktop computer within a few years as researchers figure out how to reduce the size of the model. It could be sooner because the knowledge that it is popular to reduce the size of models and disseminate them drives a lot of people to try to accomplish it. Like when the four minute mile was first run, and then suddenly many runners could do it.
knaik94 4y ago
Look into KoboldAi, it doesn't need to download the internet, just interface with it.
tiagod 4y ago
The model doesn't need all the knowledge encoded in its weights, just the ability to look things up effectively. I believe this will be available to the general public sooner rather than later.