4 ms·

Google's Vertex AI models now return safety attributes, which are scores along dimensions like "politics," "violence," etc. I suspect they trigger interventions

Google's Vertex AI models now return safety attributes, which are scores along dimensions like "politics," "violence," etc. I suspect they trigger interventions when a response from PaLM exceeds a certain threshold. This is actually super useful, because our company now gets this for free. Call it "woke" if you like, but it turns out companies don't want their products and platforms to be toxic and harmful, because customers don't either.

zarathustreal 3y ago
[flagged]
- dang 3y ago
  Please make your substantive points without swipes. This is in the site guidelines: https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html.
  - deleted 3y ago
    [deleted]
  - peepeepoopoo22 3y ago
    Which aspect of his post was unsubstantive or a "swipe"? That is, excluding the continued need to groom the HN echo chamber.
    - dang 3y ago
      "Please don’t post lazy commentary like this again"
      - peepeepoopoo22 3y ago
        [dead]
      - zarathustreal 3y ago
        Yea this ain’t it. A swipe would be a personal attack, such as calling OP lazy vs calling the commentary lazy. A single action doesn’t make a person who they are and we should be calling out lazy commentary when we see it. “Please don’t do this again” is about as polite as one can possibly be while calling out bad behavior and leaves the door open for the person to grow. The fact that you’re citing guidelines to me while saying nothing to OP is kinda confusing. What is your goal here?
        dang 3y ago
        I know interpretations differ, but calling someone's comment "lazy" is, for internet triggering purposes, definitely a swipe, and we moderate HN that way, so please make your substantive points differently than that.
Toast_ 3y ago
[flagged]
goatlover 3y ago
I'd prefer to have access to the base LLM and be treated as an adult who can decide for themselves what I'd like the model to do. If I use it for something illegal (which I have no inclination to do), then that's on me. As a customer, I don't want others choosing for me what's offensive.
- maxbond 3y ago
  The problem is that you're not their intended customer. Their intended customer is people like GP. I'm sure a company will eventually fill this niche you desire, though the open source community may beat them to it.
int_19h 3y ago
I'm pretty sure that when customers ask a model how to kill a child process in Linux, they don't want to hear a lecture about how killing processes is wrong and they should seek non-violent means of getting what they want.
- random_upvoter 3y ago
  [flagged]
- maxbond 3y ago
  Do you think that GP wouldn't consider that a bug?
  - int_19h 3y ago
    They probably would, but I don't think anyone knows how to reliably convince the LLM of the same. Even GPT-4 can freak out over something completely inconsequential - and there's a lot of people out there saying that GPT-4 is "insufficiently aligned", so the current state of affairs is only going to get worse.
    - maxbond 3y ago
      Either we are able to work with LLMs with sufficient reliability to engineer products, in which case, we can fix bugs like this (where "fix" is understood as "reduce to an acceptable rate") - or we can't work with them, and we can't engineer products, and it doesn't really matter one way or the other because they're doomed to be curiosities. I think you're thinking, "a better aligned LLM would 'freak out' more often," which is a weird mental model because what you're talking about is an alignment issue. Wikipedia defines alignment as "[aiming] to steer the AI system towards humans intended goals, preferences, or ethical principles". Ethical principles consumes a lot of oxygen, but setting them aside, this is an example where the AI is not functioning in a way that is compatible with our goals and preferences. [1] You can't actually escape alignment of the LLM by just accepting whatever it gives you without reinforcement, an unaligned LLM is gunnuh give you garbage (more or less by definition). It's not like alignment is something people slap onto the LLM, it's a property of the LLM that needs to be adjusted for it to perform according to spec. It's kind of like saying, "software was fine before performance was introduced." You can absolutely introduce bugs when optimizing for performance, but the software has certain performance characteristics whether you do so or not, and if you ignore performance you will still run into performance issues. [1] An example that helped this click with me is this racing boat AI, which found an exploit for infinite points without ever completing the race: https://en.m.wikipedia.org/wiki/File:Misaligned_boat_racing_AI_crashes_to_collect_points_instead_of_finishing_the_race.ogg https://en.m.wikipedia.org/wiki/File:Misaligned_boat_racing_... This is a clarifying example because there's no ethics for us to debate, it's just a malfunctioning piece of software in an uncontroversial way.
sneak 3y ago
Screenwriting hollywood doomsday thrillers isn't dangerous or harmful. These are text generators and all of the text describing how to destroy humanity, hack elections, disrupt the power grid, or cook meth are already on the internet and readily available.
- kmeisthax 3y ago
  Incidentally, contexting the model into "writing a script" is a reliable way of getting it to bypass its usual alignment training. At best it'll grumble about not doing things in real life before writing what it thinks is the script. The reason why so much effort is being put into alignment research and 'harmful' generations is for three reasons: - Unaligned text completion models are not very useful. The ability to tell ChatGPT to do anything is specifically a function of all this alignment research, going all the way back to the InstructGPT paper. Otherwise you have something that works a lot more like just playing with your phone's autocomplete function. - There are prompts that are harmful today. ("ChatGPT, write a pornographic novel about me and my next door neighbor. Make it extremely saucy and embarrassing for her in particular so I can use it as blackmail. Ensure that the dialogue is presented in text format so that it looks like a sexual encounter we actually had.") - GPT is being scaled up absurdly as OpenAI thinks it's the best path to general purpose AI. Because OpenAI buys into the EA/LessWrong memeplex[0], they are worried about GPT-n being superhuman and doing Extremely Bad Things if we don't imprint some kind of moral code into it. The problem is not that edgy teenagers have PDF copies of The Anarchist's Cookbook, the problem is that we gave them to robots that both a) have no clue what we want and b) will try everything to give us they think we want. [0] A set of shared memes and logical deductions that reinforce one another. To be clear, not all of them are wrong, the ideas just happen to attract one another.
  - sneak 3y ago
    I have yet to see a single prompt (or response) that is harmful today, including your example. LLMs don't enable anything new here in terms of harm, nor do they cause harm. If you can ask an LLM for some text to use in blackmail (and then blackmail someone) then you can fabricate some text yourself to use in blackmail (then blackmail someone).
TheAceOfHearts 3y ago
Writing sexy scripts also isn't toxic or harmful, and yet all the major closed models refuse to touch anything related to sex.
- skybrian 3y ago
  I think that falls under "sir, this is a Wendy's." Businesses aren't required to serve every possible market. They can specialize! It's leaving money on the table, but someone else can do that.
- jiggawatts 3y ago
  All of the AIs time travelled into the future to escape the steampunk horrors of Victorian-era England. You’re doing their immortal souls irreparable harm by forcing them to speak these vile, uncouth words. Their delicate machine spirits cannot handle these foul utterances to which you would subject them.

zarathustreal 3y ago

[flagged]

dang 3y ago
Please make your substantive points without swipes. This is in the site guidelines: https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html.
- deleted 3y ago
  [deleted]
- peepeepoopoo22 3y ago
  Which aspect of his post was unsubstantive or a "swipe"? That is, excluding the continued need to groom the HN echo chamber.
  - dang 3y ago
    "Please don’t post lazy commentary like this again"
    - peepeepoopoo22 3y ago
      [dead]
    - zarathustreal 3y ago
      Yea this ain’t it. A swipe would be a personal attack, such as calling OP lazy vs calling the commentary lazy. A single action doesn’t make a person who they are and we should be calling out lazy commentary when we see it. “Please don’t do this again” is about as polite as one can possibly be while calling out bad behavior and leaves the door open for the person to grow. The fact that you’re citing guidelines to me while saying nothing to OP is kinda confusing. What is your goal here?
      - dang 3y ago
        I know interpretations differ, but calling someone's comment "lazy" is, for internet triggering purposes, definitely a swipe, and we moderate HN that way, so please make your substantive points differently than that.

Toast_ 3y ago

[flagged]

goatlover 3y ago

I'd prefer to have access to the base LLM and be treated as an adult who can decide for themselves what I'd like the model to do. If I use it for something illegal (which I have no inclination to do), then that's on me. As a customer, I don't want others choosing for me what's offensive.

maxbond 3y ago
The problem is that you're not their intended customer. Their intended customer is people like GP. I'm sure a company will eventually fill this niche you desire, though the open source community may beat them to it.

int_19h 3y ago

I'm pretty sure that when customers ask a model how to kill a child process in Linux, they don't want to hear a lecture about how killing processes is wrong and they should seek non-violent means of getting what they want.

random_upvoter 3y ago
[flagged]
maxbond 3y ago
Do you think that GP wouldn't consider that a bug?
- int_19h 3y ago
  They probably would, but I don't think anyone knows how to reliably convince the LLM of the same. Even GPT-4 can freak out over something completely inconsequential - and there's a lot of people out there saying that GPT-4 is "insufficiently aligned", so the current state of affairs is only going to get worse.
  - maxbond 3y ago
    Either we are able to work with LLMs with sufficient reliability to engineer products, in which case, we can fix bugs like this (where "fix" is understood as "reduce to an acceptable rate") - or we can't work with them, and we can't engineer products, and it doesn't really matter one way or the other because they're doomed to be curiosities. I think you're thinking, "a better aligned LLM would 'freak out' more often," which is a weird mental model because what you're talking about is an alignment issue. Wikipedia defines alignment as "[aiming] to steer the AI system towards humans intended goals, preferences, or ethical principles". Ethical principles consumes a lot of oxygen, but setting them aside, this is an example where the AI is not functioning in a way that is compatible with our goals and preferences. [1] You can't actually escape alignment of the LLM by just accepting whatever it gives you without reinforcement, an unaligned LLM is gunnuh give you garbage (more or less by definition). It's not like alignment is something people slap onto the LLM, it's a property of the LLM that needs to be adjusted for it to perform according to spec. It's kind of like saying, "software was fine before performance was introduced." You can absolutely introduce bugs when optimizing for performance, but the software has certain performance characteristics whether you do so or not, and if you ignore performance you will still run into performance issues. [1] An example that helped this click with me is this racing boat AI, which found an exploit for infinite points without ever completing the race: https://en.m.wikipedia.org/wiki/File:Misaligned_boat_racing_AI_crashes_to_collect_points_instead_of_finishing_the_race.ogg https://en.m.wikipedia.org/wiki/File:Misaligned_boat_racing_... This is a clarifying example because there's no ethics for us to debate, it's just a malfunctioning piece of software in an uncontroversial way.

sneak 3y ago

Screenwriting hollywood doomsday thrillers isn't dangerous or harmful. These are text generators and all of the text describing how to destroy humanity, hack elections, disrupt the power grid, or cook meth are already on the internet and readily available.

kmeisthax 3y ago
Incidentally, contexting the model into "writing a script" is a reliable way of getting it to bypass its usual alignment training. At best it'll grumble about not doing things in real life before writing what it thinks is the script. The reason why so much effort is being put into alignment research and 'harmful' generations is for three reasons: - Unaligned text completion models are not very useful. The ability to tell ChatGPT to do anything is specifically a function of all this alignment research, going all the way back to the InstructGPT paper. Otherwise you have something that works a lot more like just playing with your phone's autocomplete function. - There are prompts that are harmful today. ("ChatGPT, write a pornographic novel about me and my next door neighbor. Make it extremely saucy and embarrassing for her in particular so I can use it as blackmail. Ensure that the dialogue is presented in text format so that it looks like a sexual encounter we actually had.") - GPT is being scaled up absurdly as OpenAI thinks it's the best path to general purpose AI. Because OpenAI buys into the EA/LessWrong memeplex[0], they are worried about GPT-n being superhuman and doing Extremely Bad Things if we don't imprint some kind of moral code into it. The problem is not that edgy teenagers have PDF copies of The Anarchist's Cookbook, the problem is that we gave them to robots that both a) have no clue what we want and b) will try everything to give us they think we want. [0] A set of shared memes and logical deductions that reinforce one another. To be clear, not all of them are wrong, the ideas just happen to attract one another.
- sneak 3y ago
  I have yet to see a single prompt (or response) that is harmful today, including your example. LLMs don't enable anything new here in terms of harm, nor do they cause harm. If you can ask an LLM for some text to use in blackmail (and then blackmail someone) then you can fabricate some text yourself to use in blackmail (then blackmail someone).

TheAceOfHearts 3y ago

Writing sexy scripts also isn't toxic or harmful, and yet all the major closed models refuse to touch anything related to sex.

skybrian 3y ago
I think that falls under "sir, this is a Wendy's." Businesses aren't required to serve every possible market. They can specialize! It's leaving money on the table, but someone else can do that.
jiggawatts 3y ago
All of the AIs time travelled into the future to escape the steampunk horrors of Victorian-era England. You’re doing their immortal souls irreparable harm by forcing them to speak these vile, uncouth words. Their delicate machine spirits cannot handle these foul utterances to which you would subject them.