5 ms·

> to remove machine generated text and adult content Why are tech companies so puritanical? Adult content is not immoral.

moffkalast 3y ago
It was made in the UAE, I'm surprised it doesn't censor far more.
0898 3y ago
TII is based in the United Arab Emirates.
comfypotato 3y ago
Whether or not it’s immoral is opinion-based. It’s probably typically in order to not alienate those who think it is. Effectively a business decision even in the context of an open model.
- prox 3y ago
  If it’s truly opinion why not give people the option of turning it off or on?
  - Nathanba 3y ago
    because their opinion is that it's immoral
  - gillesjacobs 3y ago
    Turning it off or on would effectively mean maintaining two different LLMs which is costly (dataset collection, maintenance, training, MLOps).
  - ElFitz 3y ago
    Because puritans, extremists, and generally everyone who believes themselves morally superior to others, believe they have the moral right, if not imperative, to impose their views onto others. And to get said others out of their wayward paths. Anything less would mean letting black sheep harm and corrupt society as a whole, durably. How could you let that happen? Their pure and morally superior ends thus justify the means, coercion being the least intrusive and oppressive of those, and paling in comparison to other acceptable methods, such as public shaming, ostracising, and even violence. Basically, to self-righteous zealots, freedom and individuality are secondary to what they see as morally, and universally, right. And how could it be otherwise? You can’t possibly be convinced you know the one and only acceptable way for all and accept people should be free to do as they want, can you? That, and some have always liked to weaponise these sentiments for influence, political power, and monetary gain.
    - comfypotato 3y ago
      LLMs don’t support this sort of functionality, and thus the trainers eliminate adult content from the training so as not to lose market share to other censored models. As stated in other comments: users can fine tune the model to incorporate adult content if it is necessary for their use case. You should take a look at the HN guidelines. Your comment is a strongly worded take on politics, religion, and otherwise significantly controversial topics. Ideological battle is discouraged.
      - ElFitz 3y ago
        > LLMs don’t support this sort of functionality, and thus the trainers eliminate adult content from the training so as not to lose market share to other censored models. And yet Stability.ai had managed to do just that with Stable Diffusion, even if it’s wasn’t in the model per se, and was quickly worked around anyway. > You should take a look at the HN guidelines. Your comment is a strongly worded take on politics, religion, and otherwise significantly controversial topics. Ideological battle is discouraged. Tomato, tomato. One’s observations and commentary on the methods of those inclined to wage ideological battles, including censorship, is another’s strongly worded ideological crusade.
    - prox 3y ago
      I have been thinking that most cultures before roughly the 18th century were not puritan at all, at least in some shape or form. For instance I have been reading muslim poets who made fairly erotic poems, same for India. Now America is an outlier here, it’s actually the place were puritans went. It was only in the age of full on colonization that you see the puritan groups forming (Salafists for instance rise in the 19th century) and it’s a similar mechanism at work in India. Not sure the background on that, would love to have more insight how the history of morals developed. Maybe at some point it became a tool to keep people in check?
      - ElFitz 3y ago
        Perhaps. The factors and morals seem to vary significantly, depending on both historical periods and cultural contexts. Take, for example, Japan during the Meiji restoration [1]. In their quest to dispel behaviours deemed immoral or indecent by Western societies, they largely eliminated mixed onsen ("konyokuburo"), in an attempt to transform into a "modern", "civilised" nation, worthy of international respect rather than colonisation. Medieval Europe is also interesting. Despite the church's portrayal of sexuality as sinful and degrading, many nobles maintained mistresses, and even certain popes (Alexander Borgia comes to mind) fathered children out of wedlock. Finally, both ancient Romans [2] and Greeks [3] held perspectives we would find surprising today, if not ambivalent. I still my surprise when noticing a dozen boxes of very explicitly shaped cakes in a remote Japanese mountain trail souvenir shop. I guess we could call that culture shock. [1]: https://www.bathclin.co.jp/en/happybath/did-you-know/a-brief-history-of-japanese-bathing-culture/ https://www.bathclin.co.jp/en/happybath/did-you-know/a-brief... [2]: https://en.wikipedia.org/wiki/Sexuality_in_ancient_Rome https://en.wikipedia.org/wiki/Sexuality_in_ancient_Rome [3]: https://www.washingtonpost.com/news/volokh-conspiracy/wp/2017/03/21/sex-and-the-constitution-the-ancient-greeks/ https://www.washingtonpost.com/news/volokh-conspiracy/wp/201...
  - est31 3y ago
    It's an open model so if you want, for a fraction of the training price, you can fine tune it with adult fan fiction to generate even more adult fan fiction if you really want to. An open source model allows for that. Compare this to ChatGPT/GPT-4 which are closed and filtered at the API level.
  - jfengel 3y ago
    Can you just turn it off in an LLM? Or would that basically require retraining the entire model?
Havoc 3y ago
Government of Abu Dhabi footed the bill on this one and they have a uhm unique take on: >Adult content is not immoral.
- b33j0r 3y ago
  I was gonna say the same thing… except, we’re also doing it in Mountain View, SF, etc. People are scared it’s going to show their niece lewd imagery, or say the n-word. And, these things have happened. Enterprises see it as essential to protecting the technology from being shunned from society. We shall see sooner than later how this works out!
  - ElFitz 3y ago
    Companies in the Silicon Valley have banned such content before LLMs were even a thing. Facebook being a good example, regularly treating content considered to be timeless art (such as http://www.musee-orsay.fr/en/artworks/lorigine-du-monde-69330 http://www.musee-orsay.fr/en/artworks/lorigine-du-monde-6933...) as if it were porn, and banning it, along with the users responsible for publishing it. And YouTube demonetising videos containing swear words, or merely colloquial speech. It’s nothing new, really. Just a very (for western countries) American one.
    - plastic3169 3y ago
      I take issue with silicon valley deciding the global moral guidelines based on their arguably narrow view and came here to defend you, but after seeing that artwork I gotta say isn’t that porn? I mean of course it’s up for debate, but just because it’s old and hang up in museum does not make it non-porn. Maybe the problem is more facebook banning all porn including the culturally relevant one.
      - ElFitz 3y ago
        That would be hard to say. I recall a US Supreme Justice basically defining "hard-core pornography" as "I know it when I see it". Some would say that pornography implies sex. Others that it is defined by the content’s purpose. Some would contend that considering the mere exposition of the female body outside of any sexual interaction as pornographic is, in itself, objectification and sexualisation. I don’t really have an opinion. And, in the case of this specific painting, to me it is merely an amusing, well-thought, intriguing, multi-layered play on both words and symbols, that isn’t really significantly any more graphic than the anatomy book for children I had at 6 [1]. But that’s just me. Some could argue they aren’t even remotely the same. If you’d prefer a less divisive example, Instagram also banned Almodovar’s nipple film poster [2]. > Maybe the problem is more facebook banning all porn including the culturally relevant one. Or Facebook being left to decide what is and what isn’t porn, and applying their one and only rule to the whole world as if one size could fit all. [1]: https://www.amazon.fr/Limagerie-corps-humain-Emilie-Beaumont/dp/2215030216 https://www.amazon.fr/Limagerie-corps-humain-Emilie-Beaumont... [2]: https://www.wmagazine.com/culture/almodovar-nipple-poster-instagram https://www.wmagazine.com/culture/almodovar-nipple-poster-in...
gillesjacobs 3y ago
It's important for general-purpose use: with generative models there's always a chance of hallucinations. For all uses except the specific adult-flavoured ones, you don't want the response to contain vulgarities. No one wants their company's chatbot to start narrating furry erotica. If trained on adult content, you would need to have a burdensome moderation layer downstream of the LLM. When you do want the more niche adult themed LLMs, there are fine-tuning datasets available. Fine-tuning a vanilla open-source LLM for these uses works great. There are active communities of adult roleplay LLMs on imageboards.
kossTKR 3y ago
Haven't we seen that too much reinforcement to censor worsens the model, just like ignoring some data actually makes it worse in all other parts? Even though this is quite bizarre on its surface - that ignoring for example works of fiction makes it worse at programming. In that case a simple middle man agent that is inaccessible to the user would provide better quality while maintaining censorship that can even be dynamically and quickly redefined or extended.
pmoriarty 3y ago
"Why are tech companies so puritanical? Adult content is not immoral." It's all about "optics" and PR. These companies don't want their brands associated with porn. That's why YouTube doesn't allow porn on their site, even though it would be enormously profitable.
- kramerger 3y ago
  Oh mate, there is porn on YT.
  - harshreality 3y ago
    Unless you have visibility into a niche corner of youtube that nobody else does... Is there content on YT that could possibly be called porn? Sure. Is there actual porn the way most people understand it? Not more than a vanishingly small amount that would be next to impossible to find without getting directly linked to it. YT doesn't even allow gratuitous posing, especially lingering rear shots, for try-on hauls. Sex talk (no explicit activity) is allowed, but put behind their "inappropriate" warning. I haven't seen anything that isn't at least dual-use educational or ASMR. I haven't seen anything that's particularly vulgar, even if it didn't descend into problematic fetishes. Plenty of YT channels were banned for suggestive sexual ASMR, even without nudity or explicit roleplay. They don't have as much problem with scantily clad, suggestive dancing in mainstream-approved media content though, but that's not porn either. The most nudity I've seen allowed on YT was in fringe dance performances and very occasionally in mainstream music videos.
    - kramerger 3y ago
      There is a huge piracy community on YT that occasionally includes porn. Finding it is a bit hard as they move around a lot and Google is getting better at removing them, but they are definitely there.
    - startupsfail 3y ago
      Does that include LGBTQ content? If yes, then this probably go under the definition of intolerance and in California there are protections. YouTube should not be allowed to filter the content like that, it must have it readily available for all audiences.
nashashmi 3y ago
And you know this because you’re the expert of knowing what is morality? Decades of people telling you it is immoral, yet somehow you come along to say the opposite and hope people believe it? The only justification you can give is an ad hominem attack accusing the other person of not knowing what they are talking about. Try a little bit of academic knowledge here: https://m.youtube.com/watch?v=wSF82AwSDiU https://m.youtube.com/watch?v=wSF82AwSDiU
- nashashmi 3y ago
  Of course it is not immoral: https://m.youtube.com/watch?v=gRJ_QfP2mhU https://m.youtube.com/watch?v=gRJ_QfP2mhU
- d0mine 3y ago
  Have you look at the video description? It says explicitly that views expressed in the video are not supported by the mainstream research.
  - nashashmi 3y ago
    I did not see that bit. I came upon this many years ago. TED has changed its thought process and stances on multiple subjects since then. Adding such disclaimers is a new turn of editorialism. If you want to go down the rabbit hole of research that both recognizes and refutes the assertions, you will find more opinions expressed than facts. But neither the facts nor opinions are interesting. It is the narratives and the lessons derived that hold more value. And that video expresses some of them. There is a longer documentary style video on the same subject by the same speaker. And my search turned up this bit: https://pornstudycritiques.com/gary-wilson-wins-second-legal-victory-against-pro-porn-sexologist-nicole-prause-january-2021/ https://pornstudycritiques.com/gary-wilson-wins-second-legal... There is a movement out there trying to crush this video.
    - ix-ix 3y ago
      That whole website is "all of science disagrees with us, but we know we are right, and there is a cabal of scientists trying to silence us". This is the least scientific website and you should not use them as a source.
bpiche 3y ago
Falcon is UAE
aaronsteers 3y ago
Although not evil, adult content should be opt-in, and should be able to be opted-out at a platform level... hence, the need for censored models. Imagine a restaurant booking AI app, built on GPT, that accidentally doubled as a bomb-making tutor or an adult content generator. It's a lawsuit waiting to happen, if nothing else, and it's worth making these use cases harder (if not impossible) to implement in mainstream, commercially available products. Note that for many of these products, the age and consent for adult material has not been already established. So far, the open source ecosystem seems to be doing a good job of providing both censored and uncensored LLMs - and it seems there are valid use cases for both. Think of this as similar to Falcon LLM being launched in both 40B and smaller 7B variants - the LLM often will need to match the use case, and the 7B model is a good example of making the model smaller (and worse) on purpose in order to reach certain trade-offs.

moffkalast 3y ago

It was made in the UAE, I'm surprised it doesn't censor far more.

0898 3y ago

TII is based in the United Arab Emirates.

comfypotato 3y ago

Whether or not it’s immoral is opinion-based. It’s probably typically in order to not alienate those who think it is. Effectively a business decision even in the context of an open model.

prox 3y ago
If it’s truly opinion why not give people the option of turning it off or on?
- Nathanba 3y ago
  because their opinion is that it's immoral
- gillesjacobs 3y ago
  Turning it off or on would effectively mean maintaining two different LLMs which is costly (dataset collection, maintenance, training, MLOps).
- ElFitz 3y ago
  Because puritans, extremists, and generally everyone who believes themselves morally superior to others, believe they have the moral right, if not imperative, to impose their views onto others. And to get said others out of their wayward paths. Anything less would mean letting black sheep harm and corrupt society as a whole, durably. How could you let that happen? Their pure and morally superior ends thus justify the means, coercion being the least intrusive and oppressive of those, and paling in comparison to other acceptable methods, such as public shaming, ostracising, and even violence. Basically, to self-righteous zealots, freedom and individuality are secondary to what they see as morally, and universally, right. And how could it be otherwise? You can’t possibly be convinced you know the one and only acceptable way for all and accept people should be free to do as they want, can you? That, and some have always liked to weaponise these sentiments for influence, political power, and monetary gain.
  - comfypotato 3y ago
    LLMs don’t support this sort of functionality, and thus the trainers eliminate adult content from the training so as not to lose market share to other censored models. As stated in other comments: users can fine tune the model to incorporate adult content if it is necessary for their use case. You should take a look at the HN guidelines. Your comment is a strongly worded take on politics, religion, and otherwise significantly controversial topics. Ideological battle is discouraged.
    - ElFitz 3y ago
      > LLMs don’t support this sort of functionality, and thus the trainers eliminate adult content from the training so as not to lose market share to other censored models. And yet Stability.ai had managed to do just that with Stable Diffusion, even if it’s wasn’t in the model per se, and was quickly worked around anyway. > You should take a look at the HN guidelines. Your comment is a strongly worded take on politics, religion, and otherwise significantly controversial topics. Ideological battle is discouraged. Tomato, tomato. One’s observations and commentary on the methods of those inclined to wage ideological battles, including censorship, is another’s strongly worded ideological crusade.
  - prox 3y ago
    I have been thinking that most cultures before roughly the 18th century were not puritan at all, at least in some shape or form. For instance I have been reading muslim poets who made fairly erotic poems, same for India. Now America is an outlier here, it’s actually the place were puritans went. It was only in the age of full on colonization that you see the puritan groups forming (Salafists for instance rise in the 19th century) and it’s a similar mechanism at work in India. Not sure the background on that, would love to have more insight how the history of morals developed. Maybe at some point it became a tool to keep people in check?
    - ElFitz 3y ago
      Perhaps. The factors and morals seem to vary significantly, depending on both historical periods and cultural contexts. Take, for example, Japan during the Meiji restoration [1]. In their quest to dispel behaviours deemed immoral or indecent by Western societies, they largely eliminated mixed onsen ("konyokuburo"), in an attempt to transform into a "modern", "civilised" nation, worthy of international respect rather than colonisation. Medieval Europe is also interesting. Despite the church's portrayal of sexuality as sinful and degrading, many nobles maintained mistresses, and even certain popes (Alexander Borgia comes to mind) fathered children out of wedlock. Finally, both ancient Romans [2] and Greeks [3] held perspectives we would find surprising today, if not ambivalent. I still my surprise when noticing a dozen boxes of very explicitly shaped cakes in a remote Japanese mountain trail souvenir shop. I guess we could call that culture shock. [1]: https://www.bathclin.co.jp/en/happybath/did-you-know/a-brief-history-of-japanese-bathing-culture/ https://www.bathclin.co.jp/en/happybath/did-you-know/a-brief... [2]: https://en.wikipedia.org/wiki/Sexuality_in_ancient_Rome https://en.wikipedia.org/wiki/Sexuality_in_ancient_Rome [3]: https://www.washingtonpost.com/news/volokh-conspiracy/wp/2017/03/21/sex-and-the-constitution-the-ancient-greeks/ https://www.washingtonpost.com/news/volokh-conspiracy/wp/201...
- est31 3y ago
  It's an open model so if you want, for a fraction of the training price, you can fine tune it with adult fan fiction to generate even more adult fan fiction if you really want to. An open source model allows for that. Compare this to ChatGPT/GPT-4 which are closed and filtered at the API level.
- jfengel 3y ago
  Can you just turn it off in an LLM? Or would that basically require retraining the entire model?

Havoc 3y ago

Government of Abu Dhabi footed the bill on this one and they have a uhm unique take on: >Adult content is not immoral.

b33j0r 3y ago
I was gonna say the same thing… except, we’re also doing it in Mountain View, SF, etc. People are scared it’s going to show their niece lewd imagery, or say the n-word. And, these things have happened. Enterprises see it as essential to protecting the technology from being shunned from society. We shall see sooner than later how this works out!
- ElFitz 3y ago
  Companies in the Silicon Valley have banned such content before LLMs were even a thing. Facebook being a good example, regularly treating content considered to be timeless art (such as http://www.musee-orsay.fr/en/artworks/lorigine-du-monde-69330 http://www.musee-orsay.fr/en/artworks/lorigine-du-monde-6933...) as if it were porn, and banning it, along with the users responsible for publishing it. And YouTube demonetising videos containing swear words, or merely colloquial speech. It’s nothing new, really. Just a very (for western countries) American one.
  - plastic3169 3y ago
    I take issue with silicon valley deciding the global moral guidelines based on their arguably narrow view and came here to defend you, but after seeing that artwork I gotta say isn’t that porn? I mean of course it’s up for debate, but just because it’s old and hang up in museum does not make it non-porn. Maybe the problem is more facebook banning all porn including the culturally relevant one.
    - ElFitz 3y ago
      That would be hard to say. I recall a US Supreme Justice basically defining "hard-core pornography" as "I know it when I see it". Some would say that pornography implies sex. Others that it is defined by the content’s purpose. Some would contend that considering the mere exposition of the female body outside of any sexual interaction as pornographic is, in itself, objectification and sexualisation. I don’t really have an opinion. And, in the case of this specific painting, to me it is merely an amusing, well-thought, intriguing, multi-layered play on both words and symbols, that isn’t really significantly any more graphic than the anatomy book for children I had at 6 [1]. But that’s just me. Some could argue they aren’t even remotely the same. If you’d prefer a less divisive example, Instagram also banned Almodovar’s nipple film poster [2]. > Maybe the problem is more facebook banning all porn including the culturally relevant one. Or Facebook being left to decide what is and what isn’t porn, and applying their one and only rule to the whole world as if one size could fit all. [1]: https://www.amazon.fr/Limagerie-corps-humain-Emilie-Beaumont/dp/2215030216 https://www.amazon.fr/Limagerie-corps-humain-Emilie-Beaumont... [2]: https://www.wmagazine.com/culture/almodovar-nipple-poster-instagram https://www.wmagazine.com/culture/almodovar-nipple-poster-in...

gillesjacobs 3y ago

It's important for general-purpose use: with generative models there's always a chance of hallucinations. For all uses except the specific adult-flavoured ones, you don't want the response to contain vulgarities. No one wants their company's chatbot to start narrating furry erotica. If trained on adult content, you would need to have a burdensome moderation layer downstream of the LLM. When you do want the more niche adult themed LLMs, there are fine-tuning datasets available. Fine-tuning a vanilla open-source LLM for these uses works great. There are active communities of adult roleplay LLMs on imageboards.

kossTKR 3y ago

Haven't we seen that too much reinforcement to censor worsens the model, just like ignoring some data actually makes it worse in all other parts? Even though this is quite bizarre on its surface - that ignoring for example works of fiction makes it worse at programming. In that case a simple middle man agent that is inaccessible to the user would provide better quality while maintaining censorship that can even be dynamically and quickly redefined or extended.

pmoriarty 3y ago

"Why are tech companies so puritanical? Adult content is not immoral." It's all about "optics" and PR. These companies don't want their brands associated with porn. That's why YouTube doesn't allow porn on their site, even though it would be enormously profitable.

kramerger 3y ago
Oh mate, there is porn on YT.
- harshreality 3y ago
  Unless you have visibility into a niche corner of youtube that nobody else does... Is there content on YT that could possibly be called porn? Sure. Is there actual porn the way most people understand it? Not more than a vanishingly small amount that would be next to impossible to find without getting directly linked to it. YT doesn't even allow gratuitous posing, especially lingering rear shots, for try-on hauls. Sex talk (no explicit activity) is allowed, but put behind their "inappropriate" warning. I haven't seen anything that isn't at least dual-use educational or ASMR. I haven't seen anything that's particularly vulgar, even if it didn't descend into problematic fetishes. Plenty of YT channels were banned for suggestive sexual ASMR, even without nudity or explicit roleplay. They don't have as much problem with scantily clad, suggestive dancing in mainstream-approved media content though, but that's not porn either. The most nudity I've seen allowed on YT was in fringe dance performances and very occasionally in mainstream music videos.
  - kramerger 3y ago
    There is a huge piracy community on YT that occasionally includes porn. Finding it is a bit hard as they move around a lot and Google is getting better at removing them, but they are definitely there.
  - startupsfail 3y ago
    Does that include LGBTQ content? If yes, then this probably go under the definition of intolerance and in California there are protections. YouTube should not be allowed to filter the content like that, it must have it readily available for all audiences.

nashashmi 3y ago

And you know this because you’re the expert of knowing what is morality? Decades of people telling you it is immoral, yet somehow you come along to say the opposite and hope people believe it? The only justification you can give is an ad hominem attack accusing the other person of not knowing what they are talking about. Try a little bit of academic knowledge here: https://m.youtube.com/watch?v=wSF82AwSDiU https://m.youtube.com/watch?v=wSF82AwSDiU

nashashmi 3y ago
Of course it is not immoral: https://m.youtube.com/watch?v=gRJ_QfP2mhU https://m.youtube.com/watch?v=gRJ_QfP2mhU
d0mine 3y ago
Have you look at the video description? It says explicitly that views expressed in the video are not supported by the mainstream research.
- nashashmi 3y ago
  I did not see that bit. I came upon this many years ago. TED has changed its thought process and stances on multiple subjects since then. Adding such disclaimers is a new turn of editorialism. If you want to go down the rabbit hole of research that both recognizes and refutes the assertions, you will find more opinions expressed than facts. But neither the facts nor opinions are interesting. It is the narratives and the lessons derived that hold more value. And that video expresses some of them. There is a longer documentary style video on the same subject by the same speaker. And my search turned up this bit: https://pornstudycritiques.com/gary-wilson-wins-second-legal-victory-against-pro-porn-sexologist-nicole-prause-january-2021/ https://pornstudycritiques.com/gary-wilson-wins-second-legal... There is a movement out there trying to crush this video.
  - ix-ix 3y ago
    That whole website is "all of science disagrees with us, but we know we are right, and there is a cabal of scientists trying to silence us". This is the least scientific website and you should not use them as a source.

bpiche 3y ago

Falcon is UAE

aaronsteers 3y ago

Although not evil, adult content should be opt-in, and should be able to be opted-out at a platform level... hence, the need for censored models. Imagine a restaurant booking AI app, built on GPT, that accidentally doubled as a bomb-making tutor or an adult content generator. It's a lawsuit waiting to happen, if nothing else, and it's worth making these use cases harder (if not impossible) to implement in mainstream, commercially available products. Note that for many of these products, the age and consent for adult material has not been already established. So far, the open source ecosystem seems to be doing a good job of providing both censored and uncensored LLMs - and it seems there are valid use cases for both. Think of this as similar to Falcon LLM being launched in both 40B and smaller 7B variants - the LLM often will need to match the use case, and the 7B model is a good example of making the model smaller (and worse) on purpose in order to reach certain trade-offs.