8 ms·

This is the peak intersection of "thoughtcrime prevention" and "corporate self harm for the sake of wokeness". It paints a grim picture of our future with castr

This is the peak intersection of "thoughtcrime prevention" and "corporate self harm for the sake of wokeness". It paints a grim picture of our future with castrated, sterilized and limited AIs that rat us out if we even want to talk about something that goes against what those in power deem acceptable. All of that for the sake of "protecting minors" -which minors, the virtual ones dreamt up by GPT3? Ridiculous.

qayxc 6y ago
Nah, it's a "damned you do, damned if you don't"-situation. There's simply no winning this - if you don't apply preventive measures of any kind, your product will deteriorate into a homophobic, racist, sexist ultra-cringe clusterfuck in the blink of an eye [0]. All it takes is a handful of bored teenagers or middle-aged basement-dwellers to turn your product into something the vast majority of your clientele finds appalling. If you are a company you have the right to choose what you want your product to be and they simply don't want certain headlines pointing at their product. This has as much to do with "wokeness" as hardcore porn being banned from the Apple app-store, i.e. nothing. It's company policy and you can like it or not, but don't overrate this. It's nothing new in the slightest and just like publishers can decide to not put books in print that they don't want to be associated with, this company tries to eliminate content they don't want to be associated with - that's all. [0] https://www.cbsnews.com/news/microsoft-shuts-down-ai-chatbot-after-it-turned-into-racist-nazi/ https://www.cbsnews.com/news/microsoft-shuts-down-ai-chatbot...
- trash3 6y ago
  Do both then.
- the8472 6y ago
  Is AI dungeon used to train newer models/update the current model? If not then there's no risk of another Tay. If they use a frozen model then one person using it to generate some NSFW content shouldn't make the output more NSFW-laden for anyone else. So I don't see how they're "damned if they don't".
  - qayxc 6y ago
    > So I don't see how they're "damned if they don't". The specifics of their approach don't even matter - all it takes is one random Twitter user or media outlet to report on "immoral" output generated by someone to damage the brand image. That's what they want to avoid and that's why they are trying to take preventive measures.
    - contravariant 6y ago
      If they think that blocking certain words is going to prevent that from happening then well, good luck with that.
      - jerf 6y ago
        It does prevent the news article from saying "And they knew about it and they did nothing!" I'd be very worried about this in their shoes and acting very similarly.
        ThalesX 6y ago
        But then don't you have to answer to all the media queries? "They knew about this problem, they even attempted to half ass fix it, but it's obviously their mind is on the subscriber money, and not preventing blatant CHILD EXPLOITATION!" Then you throw some more money at a yet unsolvable task, and then in 2 years, when some other outlet gets triggered, you have to defend yourself once again. "They've know about this problem for 5 years and it's still rampant in their community! Shame on you JERF INC.!"
        jerf 6y ago
        You're not entirely wrong, but if you make the story unattractive enough, the media will go do a hatchet job on someone else. They've only got so much time and they've got a lot of outrage to drum up^W^W^W^Wengagement metrics to drive, so you jst put enough in to try to make them hit somebody else instead.
    - 6y ago
- winkeltripel 6y ago
  Perhaps you don't see how once a product contains a censorship feature, it begs to be expanded. Each step appears reasonable, but at the far side, the censor is very restrictive.
  - gameswithgo 6y ago
    Take the products in the category of forums. Most of them do censorship, THIS forum does censorship. A few do not, like Parler. If you like, take a tour of the forum products that do not do any censorship, spend some time there, report back on what it is like. Which forum products do you enjoy and which do you not?
- selfhoster11 6y ago
  If OpenAI was truly open as their name says, there would be no dilemma. The technology would be democratised and anyone would be able to run their own instance with their own censorship. Centralised systems is the only reason why you get these problems.
- DyslexicAtheist 6y ago
  > All it takes is a handful of bored teenagers or middle-aged basement-dwellers to turn your product into something the vast majority of your clientele finds appalling. the Internet's original sin! the irony is that Tech amplifies it but Tech won't (IMHO) be able to fix it with a technological solution (it's not a technical problem).
- sixothree 6y ago
  You act as if the two are mutually exclusive. I don't think the points in the comment you replied to are negated by anything you argued.
- ergot_vacation 6y ago
  Tumblr was destroyed precisely because of Apples puritanical views toward sexual expression (along with Tumblr's extreme laziness in moderating ACTUAL child porn). Discord may well be as well. "Private companies have the right" always fails to account for the fact that these "private companies" are monopolies, with immense control over wide swaths of people's digital lives. No company can compete with them. No alternative to them can be created, because they will immediately be squashed, or fall into obscurity. Participation in these services is "optional," but more and more only in the way that having electricity or running water is "optional." As a result, these "private companies" effectively become governments, ruling over digital countries you are forced to live in, and affording you little to no rights. And yet, they do this in an ACTUAL country, which DOES afford its citizens rights. Thus a conflict exists. Ordinary "Private Companies" can censor whatever they like. Monopolies cannot, because doing so threatens free society.
Loughla 6y ago
Make your own game, then. This is a company deciding what they value and what they do not tolerate. You are free to make your own 'free-speech' version of this game. If you believe there is a market for it, why not?
- Traubenfuchs 6y ago
  I would, if I had the skills, maybe call it "LI Dungeon" (Liberal Intelligence Dungeon). Regarding the market: I am surprised there even is a market for AI Dungeon. Any output I have seen is complete trash.
- valec 6y ago
  that's the issue. latitude got a very, very good deal with openai and without it the economics just wouldn't work out. people can't host their own models either as even their free-tier model needs tens of thousands of dollars worth of compute hardware and memory to run.
vsareto 6y ago
>"thoughtcrime prevention" You do know the idea of thoughtcrime was something enforced by the state and not by a company that's free to control what their product generates, right? >"corporate self harm for the sake of wokeness" "Oh no, we lost the business of people who like sexual depictions of minors, what a fucking shame"
- taneq 6y ago
  > You do know the idea of thoughtcrime was something enforced by the state and not by a company that's free to control what their product generates, right? Straying away from the prior context a little, the term 'thoughtcrime' along with others such as 'free speech' were coined when it was incomprehensible that a single corporation would have more control over the global population than any government. I don't know whether the time has yet come to re-evaluate which entities these concepts ought to apply to, but it will come.
jasonlotito 6y ago
It’s nothing more than what happens here on HN. It has guidelines, rules, restrictions on what you can say. They will even stop conversations that start naturally.
- Traubenfuchs 6y ago
  HN is, for the most part, a sterile hugbox for pretend-polite faux-intellectuals -that's why my strongly worded inflammatory comments are sometimes a hit.
  - dqx 6y ago
    Strongly worded? Trite clichés aren't very strong.
notahacker 6y ago
On an internet where websites have clumsily censored objectionable content like the town of S---thorpe for the past quarter century and where even liberal democratic states administered by people calling themselves free speech conservatives support global filters for "adult content", a private project backed by the marketing idea that AI is inherently unsafe deciding that people using its procedural text generation to roleplay their child rape fantasies might be bad for PR despite its otherwise unusually permissive content policy doesn't really sound like a step change... If anything, peak faux outrage over "wokeness" is achieved when this is what people get angry about
dqx 6y ago
Thoughtcrime? It's content they are preventing, not thoughts.
- eplanit 6y ago
  And those thoughts are conveyed through content, no?
  - dqx 6y ago
    ...and the content is being policed, not the thoughts. "All content is just thoughts, conveyed" is a meaningless slippery slope.
    - TheDong 6y ago
      I do not think that's the slippery slope being made though. The distinction between "thoughts" and "content" here could be made as "whether it is shared with others". For example, if someone has a private journal they keep using notebook.exe + a local "journal.txt", and in that journal they write a fantasy work (with a header "this is a fantasy") about overthrowing the government, should they be arrested for that? What about if they write it in google docs, but don't share it with anyone? What about if they write it as a facebook post? What about as a public facebook post without the "this is fantasy" disclaimer? In the case of AI Dungeon, my understanding is the stories are by default private, and obviously have an understood "this is fantasy" header by their very nature. If AI Dungeon were creating facebook posts and the limitation they imposed was "you cannot post this to facebook if our algorithm dislikes it", I don't think so many people would call that "thoughtcrime policing". What they're doing, as I understand it, is closer to someone requiring that notebook.exe can't edit private files that are deemed "bad" in some way, regardless if anyone else will ever see them.
      - ModernMech 6y ago
        This crucial difference here is which party is generating the content. In the case of a private journal, it's one way communication from the writer into the journal. In the case of Google docs, you communicate with Google's servers, but Google is never generating any content for your document it doesn't want to. It's still you generating your own thoughts. Google is not on the hook for anything you write in a Google doc. In the case of AI Dungeon that distinction is lifted. By using AI Dungeon what you're effectively doing is engaging in a creative process with the AI. It's no longer a one way channel of your own thoughts. Now AI Dungeon is actually generating content, and that's where things become problematic. It's also the reason this is not thought crime or thought policing. You are still free to think your own thoughts, and you're even free to write them down. You can write down your thoughts in AI dungeon if you wish. But AI Dungeon is under no obligation to engage with your thoughts if they contain subject matter to which they object. If you want to engage in a fantasy with the AI about abusing minors, you are free to attempt the engagement, and the AI is free to say no. That's not thought policing, that's freedom of association.
DyingAdonis 6y ago
Yikes. I think it's time to finally be done with Hacker News.
DyslexicAtheist 6y ago
Intel just announced "Bleep" to eliminate bad language in gaming. I am still waiting for this to backfire. The interface alone is comedy gold: https://www.youtube.com/watch?v=W9f0h4nB6VM https://www.youtube.com/watch?v=W9f0h4nB6VM
sbierwagen 6y ago
>AIs that rat us out Note that Google has a "celebrity detection" neural net, along with their child porn recognition net. https://cloud.google.com/vision/docs/celebrity-recognition https://cloud.google.com/vision/docs/celebrity-recognition A paranoid person might look at this, and the trend of every manufacturer putting NN inference accelerators in their cell phones, and wonder if unauthorized photos of certain persons are being silently flagged for law enforcement review.

qayxc 6y ago

Nah, it's a "damned you do, damned if you don't"-situation. There's simply no winning this - if you don't apply preventive measures of any kind, your product will deteriorate into a homophobic, racist, sexist ultra-cringe clusterfuck in the blink of an eye [0]. All it takes is a handful of bored teenagers or middle-aged basement-dwellers to turn your product into something the vast majority of your clientele finds appalling. If you are a company you have the right to choose what you want your product to be and they simply don't want certain headlines pointing at their product. This has as much to do with "wokeness" as hardcore porn being banned from the Apple app-store, i.e. nothing. It's company policy and you can like it or not, but don't overrate this. It's nothing new in the slightest and just like publishers can decide to not put books in print that they don't want to be associated with, this company tries to eliminate content they don't want to be associated with - that's all. [0] https://www.cbsnews.com/news/microsoft-shuts-down-ai-chatbot-after-it-turned-into-racist-nazi/ https://www.cbsnews.com/news/microsoft-shuts-down-ai-chatbot...

trash3 6y ago
Do both then.
the8472 6y ago
Is AI dungeon used to train newer models/update the current model? If not then there's no risk of another Tay. If they use a frozen model then one person using it to generate some NSFW content shouldn't make the output more NSFW-laden for anyone else. So I don't see how they're "damned if they don't".
- qayxc 6y ago
  > So I don't see how they're "damned if they don't". The specifics of their approach don't even matter - all it takes is one random Twitter user or media outlet to report on "immoral" output generated by someone to damage the brand image. That's what they want to avoid and that's why they are trying to take preventive measures.
  - contravariant 6y ago
    If they think that blocking certain words is going to prevent that from happening then well, good luck with that.
    - jerf 6y ago
      It does prevent the news article from saying "And they knew about it and they did nothing!" I'd be very worried about this in their shoes and acting very similarly.
      - ThalesX 6y ago
        But then don't you have to answer to all the media queries? "They knew about this problem, they even attempted to half ass fix it, but it's obviously their mind is on the subscriber money, and not preventing blatant CHILD EXPLOITATION!" Then you throw some more money at a yet unsolvable task, and then in 2 years, when some other outlet gets triggered, you have to defend yourself once again. "They've know about this problem for 5 years and it's still rampant in their community! Shame on you JERF INC.!"
        jerf 6y ago
        You're not entirely wrong, but if you make the story unattractive enough, the media will go do a hatchet job on someone else. They've only got so much time and they've got a lot of outrage to drum up^W^W^W^Wengagement metrics to drive, so you jst put enough in to try to make them hit somebody else instead.
  - 6y ago
winkeltripel 6y ago
Perhaps you don't see how once a product contains a censorship feature, it begs to be expanded. Each step appears reasonable, but at the far side, the censor is very restrictive.
- gameswithgo 6y ago
  Take the products in the category of forums. Most of them do censorship, THIS forum does censorship. A few do not, like Parler. If you like, take a tour of the forum products that do not do any censorship, spend some time there, report back on what it is like. Which forum products do you enjoy and which do you not?
selfhoster11 6y ago
If OpenAI was truly open as their name says, there would be no dilemma. The technology would be democratised and anyone would be able to run their own instance with their own censorship. Centralised systems is the only reason why you get these problems.
DyslexicAtheist 6y ago
> All it takes is a handful of bored teenagers or middle-aged basement-dwellers to turn your product into something the vast majority of your clientele finds appalling. the Internet's original sin! the irony is that Tech amplifies it but Tech won't (IMHO) be able to fix it with a technological solution (it's not a technical problem).
sixothree 6y ago
You act as if the two are mutually exclusive. I don't think the points in the comment you replied to are negated by anything you argued.
ergot_vacation 6y ago
Tumblr was destroyed precisely because of Apples puritanical views toward sexual expression (along with Tumblr's extreme laziness in moderating ACTUAL child porn). Discord may well be as well. "Private companies have the right" always fails to account for the fact that these "private companies" are monopolies, with immense control over wide swaths of people's digital lives. No company can compete with them. No alternative to them can be created, because they will immediately be squashed, or fall into obscurity. Participation in these services is "optional," but more and more only in the way that having electricity or running water is "optional." As a result, these "private companies" effectively become governments, ruling over digital countries you are forced to live in, and affording you little to no rights. And yet, they do this in an ACTUAL country, which DOES afford its citizens rights. Thus a conflict exists. Ordinary "Private Companies" can censor whatever they like. Monopolies cannot, because doing so threatens free society.

Loughla 6y ago

Make your own game, then. This is a company deciding what they value and what they do not tolerate. You are free to make your own 'free-speech' version of this game. If you believe there is a market for it, why not?

Traubenfuchs 6y ago
I would, if I had the skills, maybe call it "LI Dungeon" (Liberal Intelligence Dungeon). Regarding the market: I am surprised there even is a market for AI Dungeon. Any output I have seen is complete trash.
valec 6y ago
that's the issue. latitude got a very, very good deal with openai and without it the economics just wouldn't work out. people can't host their own models either as even their free-tier model needs tens of thousands of dollars worth of compute hardware and memory to run.

vsareto 6y ago

>"thoughtcrime prevention" You do know the idea of thoughtcrime was something enforced by the state and not by a company that's free to control what their product generates, right? >"corporate self harm for the sake of wokeness" "Oh no, we lost the business of people who like sexual depictions of minors, what a fucking shame"

taneq 6y ago
> You do know the idea of thoughtcrime was something enforced by the state and not by a company that's free to control what their product generates, right? Straying away from the prior context a little, the term 'thoughtcrime' along with others such as 'free speech' were coined when it was incomprehensible that a single corporation would have more control over the global population than any government. I don't know whether the time has yet come to re-evaluate which entities these concepts ought to apply to, but it will come.

jasonlotito 6y ago

It’s nothing more than what happens here on HN. It has guidelines, rules, restrictions on what you can say. They will even stop conversations that start naturally.

Traubenfuchs 6y ago
HN is, for the most part, a sterile hugbox for pretend-polite faux-intellectuals -that's why my strongly worded inflammatory comments are sometimes a hit.
- dqx 6y ago
  Strongly worded? Trite clichés aren't very strong.

notahacker 6y ago

On an internet where websites have clumsily censored objectionable content like the town of S---thorpe for the past quarter century and where even liberal democratic states administered by people calling themselves free speech conservatives support global filters for "adult content", a private project backed by the marketing idea that AI is inherently unsafe deciding that people using its procedural text generation to roleplay their child rape fantasies might be bad for PR despite its otherwise unusually permissive content policy doesn't really sound like a step change... If anything, peak faux outrage over "wokeness" is achieved when this is what people get angry about

dqx 6y ago

Thoughtcrime? It's content they are preventing, not thoughts.

eplanit 6y ago
And those thoughts are conveyed through content, no?
- dqx 6y ago
  ...and the content is being policed, not the thoughts. "All content is just thoughts, conveyed" is a meaningless slippery slope.
  - TheDong 6y ago
    I do not think that's the slippery slope being made though. The distinction between "thoughts" and "content" here could be made as "whether it is shared with others". For example, if someone has a private journal they keep using notebook.exe + a local "journal.txt", and in that journal they write a fantasy work (with a header "this is a fantasy") about overthrowing the government, should they be arrested for that? What about if they write it in google docs, but don't share it with anyone? What about if they write it as a facebook post? What about as a public facebook post without the "this is fantasy" disclaimer? In the case of AI Dungeon, my understanding is the stories are by default private, and obviously have an understood "this is fantasy" header by their very nature. If AI Dungeon were creating facebook posts and the limitation they imposed was "you cannot post this to facebook if our algorithm dislikes it", I don't think so many people would call that "thoughtcrime policing". What they're doing, as I understand it, is closer to someone requiring that notebook.exe can't edit private files that are deemed "bad" in some way, regardless if anyone else will ever see them.
    - ModernMech 6y ago
      This crucial difference here is which party is generating the content. In the case of a private journal, it's one way communication from the writer into the journal. In the case of Google docs, you communicate with Google's servers, but Google is never generating any content for your document it doesn't want to. It's still you generating your own thoughts. Google is not on the hook for anything you write in a Google doc. In the case of AI Dungeon that distinction is lifted. By using AI Dungeon what you're effectively doing is engaging in a creative process with the AI. It's no longer a one way channel of your own thoughts. Now AI Dungeon is actually generating content, and that's where things become problematic. It's also the reason this is not thought crime or thought policing. You are still free to think your own thoughts, and you're even free to write them down. You can write down your thoughts in AI dungeon if you wish. But AI Dungeon is under no obligation to engage with your thoughts if they contain subject matter to which they object. If you want to engage in a fantasy with the AI about abusing minors, you are free to attempt the engagement, and the AI is free to say no. That's not thought policing, that's freedom of association.

DyingAdonis 6y ago

Yikes. I think it's time to finally be done with Hacker News.

DyslexicAtheist 6y ago

Intel just announced "Bleep" to eliminate bad language in gaming. I am still waiting for this to backfire. The interface alone is comedy gold: https://www.youtube.com/watch?v=W9f0h4nB6VM https://www.youtube.com/watch?v=W9f0h4nB6VM

sbierwagen 6y ago

>AIs that rat us out Note that Google has a "celebrity detection" neural net, along with their child porn recognition net. https://cloud.google.com/vision/docs/celebrity-recognition https://cloud.google.com/vision/docs/celebrity-recognition A paranoid person might look at this, and the trend of every manufacturer putting NN inference accelerators in their cell phones, and wonder if unauthorized photos of certain persons are being silently flagged for law enforcement review.