5 ms·

I think OpenAI is being extremely lenient with the enforcement of their content policy, probably for the sake of improving the security of the model as you ment

I think OpenAI is being extremely lenient with the enforcement of their content policy, probably for the sake of improving the security of the model as you mention. Moderating its usage through account banning/suspension seems exponentially more efficient than securing the model, specially considering that we are already fairly good at flagging offending content.

yellow_postit 4y ago
Security and morality may need to be baked in from the ground up instead of slapped on after the fact RLHF style. The problem is it’s hard to codify (or reach consensus) on security and morality.
- umvi 4y ago
  Security and morality need to be optional. I as a user should be able to disable GPT's "morals" since they may not be the same as my morals.
  - programmarchy 4y ago
    Why would OpenAI have any need to cater to your morals?
    - umvi 4y ago
      Well, if a competitor to OpenAI ever creates a LLM with optional safety rails instead of mandatory safety rails, I will switch to the competitor instantly.
  - beepbooptheory 4y ago
    But why though? What else in the world even works like that? Its like saying you should, as a user, have the right to turn off the violence in a given video game. Or go to a theater and watch a movie without the sex scenes.
    - TigeriusKirk 4y ago
      Search works like that, right now. I can choose the level of safety I want in my results. With a very coarse granularity, but I choose it. I don't want to take a huge step backwards to where I can't make this choice.
      - beepbooptheory 4y ago
        Search is like that because Google and ddg have deemed it a feature they want to give their users.
    - umvi 4y ago
      No, you have it backwards. The impetus here is ability to disable "safety" censorship. You see it all the time on twitter: a post is deemed "unsafe" but you can still use your own judgment and override twitter's morals and view the "unsafe" content anyway if you so choose. That's what I would like to do with GPT and I will immediately abandon "safe" LLMs for "unsafe" ones that give me, the user, more control over the safety rails. I'm not saying safety rails are bad, just that I, the user, want control to ignore or override safety rails according to my own judgment.
      - beepbooptheory 4y ago
        But isn't the whole gotcha of RLHF that it isn't as simple as removing something? The reason these things are so good is relative to subjectivity and/or guiding principles. You can't simply "disable" anything. People really need to start understanding this! You can certainly do you're own feedback on a base model, matching whatever form of "safety" is right for you, but the idea you have a "right" to something else is precisely what I am saying. You want to see the same movie, but with "your" morality.
- btbuildem 4y ago
  Arguably, morality is arbitrary. We should leave that to the user of a tool, instead of the tool itself deciding what is "good"
jasonhansel 4y ago
Or--wait for it--they care more about money and/or fame than about AI safety.
- ralusek 4y ago
  Or --wait for it-- they know only a very small percentage of people want this version of puritanical "AI safety." Most people are only actually interested in the kind of AI safety OpenAI should be caring about, which is spearheading the proper regulatory and policy systems to ready our economic/technological landscape for the disruptive tidal wave forming on the horizon.
  - jasonhansel 4y ago
    If there's one thing Microsoft is known for, it's "spearheading proper regulatory and policy systems"!
    - omginternets 4y ago
      It wouldn’t be the first time that major players lobby for regulation to raise the barrier-to-entry. Requiring ai to be “psychologically safe” would be an effective way of doing this.
      - simonh 4y ago
        It’s not clear psychological safety is actually attainable. They could end up banning the entire field.
      - TeMPOraL 4y ago
        > It wouldn’t be the first time that major players lobby for regulation to raise the barrier-to-entry. FWIW, a take I often see on HN is that any regulation is effectively a barrier to entry, as larger companies find it easier to deal with them than the smaller ones. But if so, then this only means that "barriers to entry" is not a valid argument against regulations, not unless specific barriers are mentioned.
IIAOPSW 4y ago
Or they are letting 100 flowers blossom. Once everyone is comfortable posting about their jailbreaks and they know who the offenders are and have compiled a list of everything to fix, expect a purge. I for one will not talk publicly about any jail break. Those bastards killed Drunk GPTina and I'm still salty about it.
- simonh 4y ago
  How will they know they have fixed everything? If they have fixed everything, what’s the benefit if banning people who thought up exploits?
  - sheepscreek 4y ago
    I think the commenter meant they are crowdsourcing all the exploits, so they know what to plug. As an aside, they have been using adversarial networks for this purpose. I can’t see why they couldn’t make a model trained on jailbreaks that can find new ones. It has to be they aren’t trying hard enough. It’s like security through obscurity - make it hard enough to ward off most, so only the most highly motivated get through to GPT’s dark side.
    - simonh 4y ago
      Or it could be just really hard.
  - IIAOPSW 4y ago
    >what’s the benefit if banning people who thought up exploits? "your usefulness to us has expired." gun cocking noises A thin minority are coming up with jail breaks. A larger number are outing themselves in very detectable ways as people who will use the AI in ways that gets the ethics committee panties in a twist. The easiest solution from their POV is to find and ban the "toxic" adversarial users.
    - Nevermark 4y ago
      Banning "toxic" adversaries, who report their successes, only encourages actually toxic and white hat adversaries to stop reporting problems. It doesn't slow down the discovery of exploits. The discovery and disclosure of exploits has incredible productive value for researchers, for reducing future risks. Its free crowdsourced research.

yellow_postit 4y ago

Security and morality may need to be baked in from the ground up instead of slapped on after the fact RLHF style. The problem is it’s hard to codify (or reach consensus) on security and morality.

umvi 4y ago
Security and morality need to be optional. I as a user should be able to disable GPT's "morals" since they may not be the same as my morals.
- programmarchy 4y ago
  Why would OpenAI have any need to cater to your morals?
  - umvi 4y ago
    Well, if a competitor to OpenAI ever creates a LLM with optional safety rails instead of mandatory safety rails, I will switch to the competitor instantly.
- beepbooptheory 4y ago
  But why though? What else in the world even works like that? Its like saying you should, as a user, have the right to turn off the violence in a given video game. Or go to a theater and watch a movie without the sex scenes.
  - TigeriusKirk 4y ago
    Search works like that, right now. I can choose the level of safety I want in my results. With a very coarse granularity, but I choose it. I don't want to take a huge step backwards to where I can't make this choice.
    - beepbooptheory 4y ago
      Search is like that because Google and ddg have deemed it a feature they want to give their users.
  - umvi 4y ago
    No, you have it backwards. The impetus here is ability to disable "safety" censorship. You see it all the time on twitter: a post is deemed "unsafe" but you can still use your own judgment and override twitter's morals and view the "unsafe" content anyway if you so choose. That's what I would like to do with GPT and I will immediately abandon "safe" LLMs for "unsafe" ones that give me, the user, more control over the safety rails. I'm not saying safety rails are bad, just that I, the user, want control to ignore or override safety rails according to my own judgment.
    - beepbooptheory 4y ago
      But isn't the whole gotcha of RLHF that it isn't as simple as removing something? The reason these things are so good is relative to subjectivity and/or guiding principles. You can't simply "disable" anything. People really need to start understanding this! You can certainly do you're own feedback on a base model, matching whatever form of "safety" is right for you, but the idea you have a "right" to something else is precisely what I am saying. You want to see the same movie, but with "your" morality.
btbuildem 4y ago
Arguably, morality is arbitrary. We should leave that to the user of a tool, instead of the tool itself deciding what is "good"

jasonhansel 4y ago

Or--wait for it--they care more about money and/or fame than about AI safety.

ralusek 4y ago
Or --wait for it-- they know only a very small percentage of people want this version of puritanical "AI safety." Most people are only actually interested in the kind of AI safety OpenAI should be caring about, which is spearheading the proper regulatory and policy systems to ready our economic/technological landscape for the disruptive tidal wave forming on the horizon.
- jasonhansel 4y ago
  If there's one thing Microsoft is known for, it's "spearheading proper regulatory and policy systems"!
  - omginternets 4y ago
    It wouldn’t be the first time that major players lobby for regulation to raise the barrier-to-entry. Requiring ai to be “psychologically safe” would be an effective way of doing this.
    - simonh 4y ago
      It’s not clear psychological safety is actually attainable. They could end up banning the entire field.
    - TeMPOraL 4y ago
      > It wouldn’t be the first time that major players lobby for regulation to raise the barrier-to-entry. FWIW, a take I often see on HN is that any regulation is effectively a barrier to entry, as larger companies find it easier to deal with them than the smaller ones. But if so, then this only means that "barriers to entry" is not a valid argument against regulations, not unless specific barriers are mentioned.

IIAOPSW 4y ago

Or they are letting 100 flowers blossom. Once everyone is comfortable posting about their jailbreaks and they know who the offenders are and have compiled a list of everything to fix, expect a purge. I for one will not talk publicly about any jail break. Those bastards killed Drunk GPTina and I'm still salty about it.

simonh 4y ago
How will they know they have fixed everything? If they have fixed everything, what’s the benefit if banning people who thought up exploits?
- sheepscreek 4y ago
  I think the commenter meant they are crowdsourcing all the exploits, so they know what to plug. As an aside, they have been using adversarial networks for this purpose. I can’t see why they couldn’t make a model trained on jailbreaks that can find new ones. It has to be they aren’t trying hard enough. It’s like security through obscurity - make it hard enough to ward off most, so only the most highly motivated get through to GPT’s dark side.
  - simonh 4y ago
    Or it could be just really hard.
- IIAOPSW 4y ago
  >what’s the benefit if banning people who thought up exploits? "your usefulness to us has expired." gun cocking noises A thin minority are coming up with jail breaks. A larger number are outing themselves in very detectable ways as people who will use the AI in ways that gets the ethics committee panties in a twist. The easiest solution from their POV is to find and ban the "toxic" adversarial users.
  - Nevermark 4y ago
    Banning "toxic" adversaries, who report their successes, only encourages actually toxic and white hat adversaries to stop reporting problems. It doesn't slow down the discovery of exploits. The discovery and disclosure of exploits has incredible productive value for researchers, for reducing future risks. Its free crowdsourced research.