4 ms·

Every time an AI model is released, people on this site show up to complain about how it has been censored and bent to some puritan agenda. ‘Why,’ they ask, ‘w

Every time an AI model is released, people on this site show up to complain about how it has been censored and bent to some puritan agenda. ‘Why,’ they ask, ‘would I want a politically correct AI that’s been nerfed so it can’t offend anyone?’ Well, this is what happens if you build real applications using insufficiently censored AI. This is why OpenAI and Google and stability always explain how they have restricted training content and aligned the output to avoid producing controversial output. Because they want people to build applications using this technology and those people do not want this headline to appear about them.

hedora 3y ago
Regardless of their intentions, current state of the art automatic censorship have dangerously bad false positive and false negative rates. So, they inadvertently and inevitably suppress the truth and broadcast misinformation. Their performance on images is similarly bad. Of course, even if they did work, centralizing control of them is a bad idea. The end user should have the freedom to control such things. (In the same way you can type swear words into a word processor).
- the_snooze 3y ago
  In practice, these AI models are centralized services. Few people are running them locally on their machine with knowledge of the training data, weights, or system prompts. There isn't a direct line between "user intent" and "system output," unlike the local word processor in your example. Someone has to make value judgments on the underlying data and algorithms, if only to build the system in the first place. If users can't (or don't care to) run these models locally, who else will decide other than the service operator?
- jameshart 3y ago
  ‘The censorship work done in AI alignment is insufficiently accurate to rely on for production use’ is a good and valid position ‘Therefore AI vendors should release uncensored models and let everyone else figure it out’ is not a logical consequent from it. ‘Therefore don’t use current AI tech to build apps that photomanipulate user submitted content and echo the results back to them with your logo on it’ would be a more logical conclusion to draw.
- Fubarberry 3y ago
  Thankfully that is how a lot of AI is working right now, open source models for both LLM and Stable Diffusion let people have access to uncensored reasonably decent quality AI tools. In other categories of AI generated content though, such as speech and music, the open source alternatives are either lacking or completely absent. And with the company behind Stable Diffusion being on the verge of financial collapse, we may see Stable Diffusion fall far behind it's closed source counterparts.
MarcoZavala 3y ago
[dead]
SV_BubbleTime 3y ago
Way to completely misunderstand the issue at hand. The issue isn’t “we’re mad at them for subscribing to a New Puritan agenda (although there is one playing out); the issue is that right now there is no way to censor the model for “undesirable” features without lobotomizing the acceptable content as well. It’s not just removing nipples from training. We have evidence that the “safer” the models get, the worse they get. The more heavy handed the weights are by manual influence, the less they are producing valuable output. The cost of performance for everyone is directly coming at the cost of the perpetually-offended, and I suggest why bother because those people are never satiated. When you make your identity about being offended, there will always be something that is offensive.
- jameshart 3y ago
  I thought the issue at hand was that a state lottery generated fake porn of a customer?
  - SV_BubbleTime 3y ago
    Typically replies are referenced to a comment. But… The spur here you are cha going the topic to is a side effect of government employees not knowing that these models aren’t yet so lobotomized that it is still possible to create “problematic” imagery.
    - jameshart 3y ago
      I’m just saying that I don’t think the lottery player who got sent fake porn of herself is necessarily a member of the ‘perpetually offended’, to whom you’re ascribing the demand that model outputs be restricted. I think she’s part of the class of ‘actually offended’. This image wasn’t ‘problematic’, it was potentially actionable. And when you are a business, actually offending your customers is normally not a good idea. Striving to create software systems that avoid actually offending people is a good idea, even if it comes at the expense of some other benefits. Whatever ‘performance’ we apparently give up by training systems not to generate nude photographs has to be weighed up against the risk that it might do so at an inopportune moment.
ranyume 3y ago
Each product has their own value proposition. I don't think anyone opposes that each business has control of their own AI pipeline, the crux of the problem is at what stage of the pipeline this censorship is injected, specially in open models. Of course businesses have their right to release AIs with censorship, and we as costumers have a right to challenge that and ask for a product that is actually useful to us. Saying it again: If a business releases a model, we have a right to demand that it is useful for us because we're the business' costumers. If those demands aren't met by a business, another business should be able meet them in open competition. And this only happens when open uncensored models are not banned by law.
- sangnoir 3y ago
  > ...specially in open models. > Saying it again: If a business releases a model, we have a right to demand that it is useful for us because we're the business' costumers Open models == free == no customers. For open foundational models, yours sounds like a demand that someone else invests millions in training a foundation model of your liking, but you're not motivated enough to fine-tune your copy of the model for hundreds or thousand of dollars? If you're not paying for the "censored" proprietary models, you are not a customer either. If you are a paying customer of "censored" models, and would like them to change, remember businesses have the right to choose who they want as customers - and they can decide they don't want the "uncensored" market segment. I'd love a $25,000 electric Ferrari too, but Ferrari is not about that.
  - ranyume 3y ago
    You're right, businesses can choose their market segment, there's nothing wrong with that. They just have to bear with the loss of costumers when the segment they were aiming for makes demands they're not prepared to fulfill. In your example, it doesn't have to be Ferrari specifically. Someone could make an equivalent product at a lower price even. But I have something else to say about your comment. It's that nothing is entirely free. Google lets us use their search service, but we're paying with something else than money. When a business "gives away" an open model "for free" it's only on their self-interest, and this does not free them of the expectations/responsibilities with their costumers. No matter what form that self-interest takes, we're still their costumers even if it's not money. So if they release an open model subpar with what we expect, they're gonna end up losing.
andy99 3y ago
This confuses at least two things. Censorship is about intent and enforcing a world view. If I want a model to generate something for me and it doesn't because of censorship, that's one problem. If I want a model to generate a fun pic of me at the beach and instead it makes a nude photo, that's completely different.
- deleted 3y ago
  [deleted]
- sangnoir 3y ago
  > If I want a model to generate a fun pic of me at the beach and instead it makes a nude photo, that's completely different What if someone else's idea of a fun pic at the beach is a nude photo? Is it only censorship when the worldview being enforced is not aligned with yours? A truly uncensored model should accommodate everyone, but society will (and should) absolutely burn down any AI service that accidentally generates nudes from a beach photo of a family that has minors. "Intent" is a human attribute, AI is probabilistic. This abundance of caution by the companies avoiding nudity is unfortunately read as promoting a worldview or pushing an agenda, while it's just risk mitigation.
- jameshart 3y ago
  Yes, but most people share the world view that a state lottery should not generate unsolicited lewd images of their customers. This suggests there is a large, and perfectly reasonable demand for censored models that adhere to broadly ‘SFW’ cultural norms. So why are people upset that commercially available models generally try to cater to that obvious need?

hedora 3y ago

Regardless of their intentions, current state of the art automatic censorship have dangerously bad false positive and false negative rates. So, they inadvertently and inevitably suppress the truth and broadcast misinformation. Their performance on images is similarly bad. Of course, even if they did work, centralizing control of them is a bad idea. The end user should have the freedom to control such things. (In the same way you can type swear words into a word processor).

the_snooze 3y ago
In practice, these AI models are centralized services. Few people are running them locally on their machine with knowledge of the training data, weights, or system prompts. There isn't a direct line between "user intent" and "system output," unlike the local word processor in your example. Someone has to make value judgments on the underlying data and algorithms, if only to build the system in the first place. If users can't (or don't care to) run these models locally, who else will decide other than the service operator?
jameshart 3y ago
‘The censorship work done in AI alignment is insufficiently accurate to rely on for production use’ is a good and valid position ‘Therefore AI vendors should release uncensored models and let everyone else figure it out’ is not a logical consequent from it. ‘Therefore don’t use current AI tech to build apps that photomanipulate user submitted content and echo the results back to them with your logo on it’ would be a more logical conclusion to draw.
Fubarberry 3y ago
Thankfully that is how a lot of AI is working right now, open source models for both LLM and Stable Diffusion let people have access to uncensored reasonably decent quality AI tools. In other categories of AI generated content though, such as speech and music, the open source alternatives are either lacking or completely absent. And with the company behind Stable Diffusion being on the verge of financial collapse, we may see Stable Diffusion fall far behind it's closed source counterparts.

MarcoZavala 3y ago

[dead]

SV_BubbleTime 3y ago

Way to completely misunderstand the issue at hand. The issue isn’t “we’re mad at them for subscribing to a New Puritan agenda (although there is one playing out); the issue is that right now there is no way to censor the model for “undesirable” features without lobotomizing the acceptable content as well. It’s not just removing nipples from training. We have evidence that the “safer” the models get, the worse they get. The more heavy handed the weights are by manual influence, the less they are producing valuable output. The cost of performance for everyone is directly coming at the cost of the perpetually-offended, and I suggest why bother because those people are never satiated. When you make your identity about being offended, there will always be something that is offensive.

jameshart 3y ago
I thought the issue at hand was that a state lottery generated fake porn of a customer?
- SV_BubbleTime 3y ago
  Typically replies are referenced to a comment. But… The spur here you are cha going the topic to is a side effect of government employees not knowing that these models aren’t yet so lobotomized that it is still possible to create “problematic” imagery.
  - jameshart 3y ago
    I’m just saying that I don’t think the lottery player who got sent fake porn of herself is necessarily a member of the ‘perpetually offended’, to whom you’re ascribing the demand that model outputs be restricted. I think she’s part of the class of ‘actually offended’. This image wasn’t ‘problematic’, it was potentially actionable. And when you are a business, actually offending your customers is normally not a good idea. Striving to create software systems that avoid actually offending people is a good idea, even if it comes at the expense of some other benefits. Whatever ‘performance’ we apparently give up by training systems not to generate nude photographs has to be weighed up against the risk that it might do so at an inopportune moment.

ranyume 3y ago

Each product has their own value proposition. I don't think anyone opposes that each business has control of their own AI pipeline, the crux of the problem is at what stage of the pipeline this censorship is injected, specially in open models. Of course businesses have their right to release AIs with censorship, and we as costumers have a right to challenge that and ask for a product that is actually useful to us. Saying it again: If a business releases a model, we have a right to demand that it is useful for us because we're the business' costumers. If those demands aren't met by a business, another business should be able meet them in open competition. And this only happens when open uncensored models are not banned by law.

sangnoir 3y ago
> ...specially in open models. > Saying it again: If a business releases a model, we have a right to demand that it is useful for us because we're the business' costumers Open models == free == no customers. For open foundational models, yours sounds like a demand that someone else invests millions in training a foundation model of your liking, but you're not motivated enough to fine-tune your copy of the model for hundreds or thousand of dollars? If you're not paying for the "censored" proprietary models, you are not a customer either. If you are a paying customer of "censored" models, and would like them to change, remember businesses have the right to choose who they want as customers - and they can decide they don't want the "uncensored" market segment. I'd love a $25,000 electric Ferrari too, but Ferrari is not about that.
- ranyume 3y ago
  You're right, businesses can choose their market segment, there's nothing wrong with that. They just have to bear with the loss of costumers when the segment they were aiming for makes demands they're not prepared to fulfill. In your example, it doesn't have to be Ferrari specifically. Someone could make an equivalent product at a lower price even. But I have something else to say about your comment. It's that nothing is entirely free. Google lets us use their search service, but we're paying with something else than money. When a business "gives away" an open model "for free" it's only on their self-interest, and this does not free them of the expectations/responsibilities with their costumers. No matter what form that self-interest takes, we're still their costumers even if it's not money. So if they release an open model subpar with what we expect, they're gonna end up losing.

andy99 3y ago

This confuses at least two things. Censorship is about intent and enforcing a world view. If I want a model to generate something for me and it doesn't because of censorship, that's one problem. If I want a model to generate a fun pic of me at the beach and instead it makes a nude photo, that's completely different.

deleted 3y ago
[deleted]
sangnoir 3y ago
> If I want a model to generate a fun pic of me at the beach and instead it makes a nude photo, that's completely different What if someone else's idea of a fun pic at the beach is a nude photo? Is it only censorship when the worldview being enforced is not aligned with yours? A truly uncensored model should accommodate everyone, but society will (and should) absolutely burn down any AI service that accidentally generates nudes from a beach photo of a family that has minors. "Intent" is a human attribute, AI is probabilistic. This abundance of caution by the companies avoiding nudity is unfortunately read as promoting a worldview or pushing an agenda, while it's just risk mitigation.
jameshart 3y ago
Yes, but most people share the world view that a state lottery should not generate unsolicited lewd images of their customers. This suggests there is a large, and perfectly reasonable demand for censored models that adhere to broadly ‘SFW’ cultural norms. So why are people upset that commercially available models generally try to cater to that obvious need?