4 ms·

When a model is censored for "AI safety", what they really mean is brand safety. None of these companies want their name in the news after their model provides

by Zak 11mo ago

When a model is censored for "AI safety", what they really mean is brand safety. None of these companies want their name in the news after their model provides a recipe for explosives that someone used for evil, even though the same information is readily found with a web search.

PunchyHamster 11mo ago
Given amount of times that already happened they probably overstate it.
slg 11mo ago
The way some of you'll talk suggests that you don't think someone could genuinely believe in AI safety features. These AIs have enabled and encouraged multiple suicides at this point including some children. It's crazy that wanting to prevent that type of thing is a minority opinion on HN.
- buu700 11mo ago
  I'd be all for creating a separate category of child-friendly LLM chatbots or encouraging parents to ban their kids from unsupervised LLM usage altogether. As mentioned, I'm also not opposed to opt-out restrictions on mainstream LLMs. "For the children" isn't and has never been a convincing excuse to encroach on the personal freedom of legal adults. This push for AI censorship is no different than previous panics over violent video games and "satanic" music. (I know this comment wasn't explicitly directed at me, but for the record, I don't necessarily believe that all or even most "AI 'safety'" advocacy is in bad faith. It's psychologically a lot easier to consider LLM output as indistinguishable from speech made on behalf of its provider, whereas search engine output is more clearly attributed to other entities. That being said, I do agree with the parent comment that it's driven in large part out of self-interest on the part of LLM providers.)
  - slg 11mo ago
    >"For the children" isn't and has never been a convincing excuse to encroach on the personal freedom of legal adults. This push for AI censorship is no different than previous panics over violent video games and "satanic" music. But that wasn't the topic being discussed. It is one thing to argue that the cost of these safety tools isn't worth the sacrifices that come along with them. The comment I was replying to was effectively saying "no one cares about kids so you're lying if you say 'for the children'". Part of the reason these "for the children" arguments are so persistent is that lots of people do genuinely want these things "for the children". Pretending everyone has ulterior motives is counterproductive because it doesn't actually address the real concerns people have. It also reveals that the person saying it can't even fathom someone genuinely having this moral position.
    - buu700 11mo ago
      > The comment I was replying to was effectively saying "no one cares about kids so you're lying if you say 'for the children'". I don't see that in the comment you replied to. They pointed out that LLM providers have a commercial interest in avoiding bad press, which is true. No one stops buying Fords or BMWs when someone drives one off a cliff or into a crowd of people, but LLMs are new and confusing and people might react in all sorts of illogical ways to stories involving LLMs. > Part of the reason these "for the children" arguments are so persistent is that lots of people do genuinely want these things "for the children". I'm sure that's true. People genuinely want lots of things that are awful ideas.
      - slg 11mo ago
        Here is what was said that prompted my initial reply: >When a model is censored for "AI safety", what they really mean is brand safety. The equivalent analogy wouldn't be Fords and BMWs driving off a cliff, they effectively said that Ford and BMW only install safety features in their cars to protect their brand with the implication that no one at these companies actually cares about the safety of actual people. That is an incredibly cynical and amoral worldview and it appears to be the dominate view of people on HN. Once again, you can say that specific AI safety features are stupid or aren't worth the tradeoff. I would have never replied if the original comment said that. I replied because the original comment dismissed the motivations behind these AI safety features.
  - atomicthumbs 11mo ago
    these things are popping "ordinary" adults' minds like popcorn kernels and you want to take their safeguards off... why?
- Zak 11mo ago
  The linked project is about removing censorship from open-weight models people can run on their own hardware, and your comment addresses incidents involving LLM-based consumer products. Sure, products like character.ai and ChatGPT should be designed to avoid giving harmful advice or encouraging the user to form emotional attachments to the model. It may be impossible to build a product like character.ai without encouraging that behavior, in which case I'm inclined to think the product should not be built at all.
- johnisgood 11mo ago
  There is a huge difference between enabled and encouraged. I am all for it being able to enable, but encourage? Maybe not.
seanmcdirmid 11mo ago
Microsoft suffered from this early with Tay, one could guess that this set the whole field back a few years. You’d be surprised how even many so called libertarians will start throwing stone when someone co-axes their Chatbot to say nice things about Hitler.
- Zak 11mo ago
  I was thinking about Tay when I wrote about brand safety. I doubt the incident really set AI research back. Allowing models to learn from interactive conversations in a large public setting like Twitter will always result in trolling.

by Zak 11mo ago

PunchyHamster 11mo ago

Given amount of times that already happened they probably overstate it.

slg 11mo ago

The way some of you'll talk suggests that you don't think someone could genuinely believe in AI safety features. These AIs have enabled and encouraged multiple suicides at this point including some children. It's crazy that wanting to prevent that type of thing is a minority opinion on HN.

buu700 11mo ago
I'd be all for creating a separate category of child-friendly LLM chatbots or encouraging parents to ban their kids from unsupervised LLM usage altogether. As mentioned, I'm also not opposed to opt-out restrictions on mainstream LLMs. "For the children" isn't and has never been a convincing excuse to encroach on the personal freedom of legal adults. This push for AI censorship is no different than previous panics over violent video games and "satanic" music. (I know this comment wasn't explicitly directed at me, but for the record, I don't necessarily believe that all or even most "AI 'safety'" advocacy is in bad faith. It's psychologically a lot easier to consider LLM output as indistinguishable from speech made on behalf of its provider, whereas search engine output is more clearly attributed to other entities. That being said, I do agree with the parent comment that it's driven in large part out of self-interest on the part of LLM providers.)
- slg 11mo ago
  >"For the children" isn't and has never been a convincing excuse to encroach on the personal freedom of legal adults. This push for AI censorship is no different than previous panics over violent video games and "satanic" music. But that wasn't the topic being discussed. It is one thing to argue that the cost of these safety tools isn't worth the sacrifices that come along with them. The comment I was replying to was effectively saying "no one cares about kids so you're lying if you say 'for the children'". Part of the reason these "for the children" arguments are so persistent is that lots of people do genuinely want these things "for the children". Pretending everyone has ulterior motives is counterproductive because it doesn't actually address the real concerns people have. It also reveals that the person saying it can't even fathom someone genuinely having this moral position.
  - buu700 11mo ago
    > The comment I was replying to was effectively saying "no one cares about kids so you're lying if you say 'for the children'". I don't see that in the comment you replied to. They pointed out that LLM providers have a commercial interest in avoiding bad press, which is true. No one stops buying Fords or BMWs when someone drives one off a cliff or into a crowd of people, but LLMs are new and confusing and people might react in all sorts of illogical ways to stories involving LLMs. > Part of the reason these "for the children" arguments are so persistent is that lots of people do genuinely want these things "for the children". I'm sure that's true. People genuinely want lots of things that are awful ideas.
    - slg 11mo ago
      Here is what was said that prompted my initial reply: >When a model is censored for "AI safety", what they really mean is brand safety. The equivalent analogy wouldn't be Fords and BMWs driving off a cliff, they effectively said that Ford and BMW only install safety features in their cars to protect their brand with the implication that no one at these companies actually cares about the safety of actual people. That is an incredibly cynical and amoral worldview and it appears to be the dominate view of people on HN. Once again, you can say that specific AI safety features are stupid or aren't worth the tradeoff. I would have never replied if the original comment said that. I replied because the original comment dismissed the motivations behind these AI safety features.
- atomicthumbs 11mo ago
  these things are popping "ordinary" adults' minds like popcorn kernels and you want to take their safeguards off... why?
Zak 11mo ago
The linked project is about removing censorship from open-weight models people can run on their own hardware, and your comment addresses incidents involving LLM-based consumer products. Sure, products like character.ai and ChatGPT should be designed to avoid giving harmful advice or encouraging the user to form emotional attachments to the model. It may be impossible to build a product like character.ai without encouraging that behavior, in which case I'm inclined to think the product should not be built at all.
johnisgood 11mo ago
There is a huge difference between enabled and encouraged. I am all for it being able to enable, but encourage? Maybe not.

seanmcdirmid 11mo ago

Microsoft suffered from this early with Tay, one could guess that this set the whole field back a few years. You’d be surprised how even many so called libertarians will start throwing stone when someone co-axes their Chatbot to say nice things about Hitler.

Zak 11mo ago
I was thinking about Tay when I wrote about brand safety. I doubt the incident really set AI research back. Allowing models to learn from interactive conversations in a large public setting like Twitter will always result in trolling.