16 ms·

AI Dungeon will block certain words, review content flagged as inappropriate

110 points by rahidz 5y ago

Traubenfuchs 5y ago
This is the peak intersection of "thoughtcrime prevention" and "corporate self harm for the sake of wokeness". It paints a grim picture of our future with castrated, sterilized and limited AIs that rat us out if we even want to talk about something that goes against what those in power deem acceptable. All of that for the sake of "protecting minors" -which minors, the virtual ones dreamt up by GPT3? Ridiculous.
- qayxc 5y ago
  Nah, it's a "damned you do, damned if you don't"-situation. There's simply no winning this - if you don't apply preventive measures of any kind, your product will deteriorate into a homophobic, racist, sexist ultra-cringe clusterfuck in the blink of an eye [0]. All it takes is a handful of bored teenagers or middle-aged basement-dwellers to turn your product into something the vast majority of your clientele finds appalling. If you are a company you have the right to choose what you want your product to be and they simply don't want certain headlines pointing at their product. This has as much to do with "wokeness" as hardcore porn being banned from the Apple app-store, i.e. nothing. It's company policy and you can like it or not, but don't overrate this. It's nothing new in the slightest and just like publishers can decide to not put books in print that they don't want to be associated with, this company tries to eliminate content they don't want to be associated with - that's all. [0] https://www.cbsnews.com/news/microsoft-shuts-down-ai-chatbot-after-it-turned-into-racist-nazi/ https://www.cbsnews.com/news/microsoft-shuts-down-ai-chatbot...
  - trash3 5y ago
    Do both then.
  - the8472 5y ago
    Is AI dungeon used to train newer models/update the current model? If not then there's no risk of another Tay. If they use a frozen model then one person using it to generate some NSFW content shouldn't make the output more NSFW-laden for anyone else. So I don't see how they're "damned if they don't".
    - qayxc 5y ago
      > So I don't see how they're "damned if they don't". The specifics of their approach don't even matter - all it takes is one random Twitter user or media outlet to report on "immoral" output generated by someone to damage the brand image. That's what they want to avoid and that's why they are trying to take preventive measures.
      - contravariant 5y ago
        If they think that blocking certain words is going to prevent that from happening then well, good luck with that.
        jerf 5y ago
        It does prevent the news article from saying "And they knew about it and they did nothing!" I'd be very worried about this in their shoes and acting very similarly.
        ThalesX 5y ago
        But then don't you have to answer to all the media queries? "They knew about this problem, they even attempted to half ass fix it, but it's obviously their mind is on the subscriber money, and not preventing blatant CHILD EXPLOITATION!" Then you throw some more money at a yet unsolvable task, and then in 2 years, when some other outlet gets triggered, you have to defend yourself once again. "They've know about this problem for 5 years and it's still rampant in their community! Shame on you JERF INC.!"
        5y ago
  - winkeltripel 5y ago
    Perhaps you don't see how once a product contains a censorship feature, it begs to be expanded. Each step appears reasonable, but at the far side, the censor is very restrictive.
    - gameswithgo 5y ago
      Take the products in the category of forums. Most of them do censorship, THIS forum does censorship. A few do not, like Parler. If you like, take a tour of the forum products that do not do any censorship, spend some time there, report back on what it is like. Which forum products do you enjoy and which do you not?
  - selfhoster11 5y ago
    If OpenAI was truly open as their name says, there would be no dilemma. The technology would be democratised and anyone would be able to run their own instance with their own censorship. Centralised systems is the only reason why you get these problems.
  - DyslexicAtheist 5y ago
    > All it takes is a handful of bored teenagers or middle-aged basement-dwellers to turn your product into something the vast majority of your clientele finds appalling. the Internet's original sin! the irony is that Tech amplifies it but Tech won't (IMHO) be able to fix it with a technological solution (it's not a technical problem).
  - sixothree 5y ago
    You act as if the two are mutually exclusive. I don't think the points in the comment you replied to are negated by anything you argued.
  - ergot_vacation 5y ago
    Tumblr was destroyed precisely because of Apples puritanical views toward sexual expression (along with Tumblr's extreme laziness in moderating ACTUAL child porn). Discord may well be as well. "Private companies have the right" always fails to account for the fact that these "private companies" are monopolies, with immense control over wide swaths of people's digital lives. No company can compete with them. No alternative to them can be created, because they will immediately be squashed, or fall into obscurity. Participation in these services is "optional," but more and more only in the way that having electricity or running water is "optional." As a result, these "private companies" effectively become governments, ruling over digital countries you are forced to live in, and affording you little to no rights. And yet, they do this in an ACTUAL country, which DOES afford its citizens rights. Thus a conflict exists. Ordinary "Private Companies" can censor whatever they like. Monopolies cannot, because doing so threatens free society.
- Loughla 5y ago
  Make your own game, then. This is a company deciding what they value and what they do not tolerate. You are free to make your own 'free-speech' version of this game. If you believe there is a market for it, why not?
  - Traubenfuchs 5y ago
    I would, if I had the skills, maybe call it "LI Dungeon" (Liberal Intelligence Dungeon). Regarding the market: I am surprised there even is a market for AI Dungeon. Any output I have seen is complete trash.
  - valec 5y ago
    that's the issue. latitude got a very, very good deal with openai and without it the economics just wouldn't work out. people can't host their own models either as even their free-tier model needs tens of thousands of dollars worth of compute hardware and memory to run.
- vsareto 5y ago
  >"thoughtcrime prevention" You do know the idea of thoughtcrime was something enforced by the state and not by a company that's free to control what their product generates, right? >"corporate self harm for the sake of wokeness" "Oh no, we lost the business of people who like sexual depictions of minors, what a fucking shame"
  - taneq 5y ago
    > You do know the idea of thoughtcrime was something enforced by the state and not by a company that's free to control what their product generates, right? Straying away from the prior context a little, the term 'thoughtcrime' along with others such as 'free speech' were coined when it was incomprehensible that a single corporation would have more control over the global population than any government. I don't know whether the time has yet come to re-evaluate which entities these concepts ought to apply to, but it will come.
- jasonlotito 5y ago
  It’s nothing more than what happens here on HN. It has guidelines, rules, restrictions on what you can say. They will even stop conversations that start naturally.
  - Traubenfuchs 5y ago
    HN is, for the most part, a sterile hugbox for pretend-polite faux-intellectuals -that's why my strongly worded inflammatory comments are sometimes a hit.
    - dqx 5y ago
      Strongly worded? Trite clichés aren't very strong.
- notahacker 5y ago
  On an internet where websites have clumsily censored objectionable content like the town of S---thorpe for the past quarter century and where even liberal democratic states administered by people calling themselves free speech conservatives support global filters for "adult content", a private project backed by the marketing idea that AI is inherently unsafe deciding that people using its procedural text generation to roleplay their child rape fantasies might be bad for PR despite its otherwise unusually permissive content policy doesn't really sound like a step change... If anything, peak faux outrage over "wokeness" is achieved when this is what people get angry about
- dqx 5y ago
  Thoughtcrime? It's content they are preventing, not thoughts.
  - eplanit 5y ago
    And those thoughts are conveyed through content, no?
    - dqx 5y ago
      ...and the content is being policed, not the thoughts. "All content is just thoughts, conveyed" is a meaningless slippery slope.
      - TheDong 5y ago
        I do not think that's the slippery slope being made though. The distinction between "thoughts" and "content" here could be made as "whether it is shared with others". For example, if someone has a private journal they keep using notebook.exe + a local "journal.txt", and in that journal they write a fantasy work (with a header "this is a fantasy") about overthrowing the government, should they be arrested for that? What about if they write it in google docs, but don't share it with anyone? What about if they write it as a facebook post? What about as a public facebook post without the "this is fantasy" disclaimer? In the case of AI Dungeon, my understanding is the stories are by default private, and obviously have an understood "this is fantasy" header by their very nature. If AI Dungeon were creating facebook posts and the limitation they imposed was "you cannot post this to facebook if our algorithm dislikes it", I don't think so many people would call that "thoughtcrime policing". What they're doing, as I understand it, is closer to someone requiring that notebook.exe can't edit private files that are deemed "bad" in some way, regardless if anyone else will ever see them.
        ModernMech 5y ago
        This crucial difference here is which party is generating the content. In the case of a private journal, it's one way communication from the writer into the journal. In the case of Google docs, you communicate with Google's servers, but Google is never generating any content for your document it doesn't want to. It's still you generating your own thoughts. Google is not on the hook for anything you write in a Google doc. In the case of AI Dungeon that distinction is lifted. By using AI Dungeon what you're effectively doing is engaging in a creative process with the AI. It's no longer a one way channel of your own thoughts. Now AI Dungeon is actually generating content, and that's where things become problematic. It's also the reason this is not thought crime or thought policing. You are still free to think your own thoughts, and you're even free to write them down. You can write down your thoughts in AI dungeon if you wish. But AI Dungeon is under no obligation to engage with your thoughts if they contain subject matter to which they object. If you want to engage in a fantasy with the AI about abusing minors, you are free to attempt the engagement, and the AI is free to say no. That's not thought policing, that's freedom of association.
- DyingAdonis 5y ago
  Yikes. I think it's time to finally be done with Hacker News.
- DyslexicAtheist 5y ago
  Intel just announced "Bleep" to eliminate bad language in gaming. I am still waiting for this to backfire. The interface alone is comedy gold: https://www.youtube.com/watch?v=W9f0h4nB6VM https://www.youtube.com/watch?v=W9f0h4nB6VM
- sbierwagen 5y ago
  >AIs that rat us out Note that Google has a "celebrity detection" neural net, along with their child porn recognition net. https://cloud.google.com/vision/docs/celebrity-recognition https://cloud.google.com/vision/docs/celebrity-recognition A paranoid person might look at this, and the trend of every manufacturer putting NN inference accelerators in their cell phones, and wonder if unauthorized photos of certain persons are being silently flagged for law enforcement review.
teddyh 5y ago
> What kind of content are you preventing? > This test is focused on preventing the use of AI Dungeon to create child sexual abuse material. How can you even begin to argue against this? It’s one of the horsemen of the infocalypse; any counterarguments are doomed.
- gambiting 5y ago
  By pointing out that traditionally this sort of thing inveitably follows the same pattern. You start with saying you're protecting the children, because no one can argue with it. Then ban any content that's potentially offensive to anybody. Or filter it so only "one way" is acceptable. Next AI dungeon won't be able to generate a character called Jesus or Prophet Muhammad because might be offensive. Then of course anything that might be interpreted as leaning liberal/conservative, depending on what the authors think is "correct". Then eventually you can't create a character named after a politician because "they want to keep the game clean and free of politics". Obviously I'm not equating any of this with CP - but I wish someone had the energy to stand up to it and say "look, you're censoring AI. It's dumb". But of course no one will because being accused as defender of CP is one of the worst things that can happen in any online discussion.
  - ModernMech 5y ago
    Can you give an example where this slippery slope occurred? For example, in Fable III and Skyrim you can't kill children, or in Morrowind there are no children at all, but you can still name your character Jesus. You say this pattern is inevitable, but if that were the case we'd see it everywhere that limitations are put in place in the name of protecting children.
    - mulander 5y ago
      In Fallout one and two, the main character had the option to kill NPC child characters. The game went so far as to mark you with a specific perk "Childkiller"[1] after performing that act. From the linked fandom wiki, quote by Tim Cain about the subject: > This led to the child killing controversy. We said look, we're going to have kids in the game; you shoot them, it's a huge penalty to karma, you're really disliked, there are places that won't sell to you, people will shoot you on sight, and we thought people can decide what they want to do. [...] This of course contributed to our M-rating, however, Europe said "no". They wouldn't even sell the game if there were children in the game. We didn't have time to rewrite all the quests, we just deleted kids off the disc. [1] - https://fallout.fandom.com/wiki/Childkiller https://fallout.fandom.com/wiki/Childkiller
inopinatus 5y ago
Good grief: "Latitude reviews content flagged by the model" - or, as it was put in another forum: every time the AI flags your game, a Latitude employee will be personally reading your private content. The key reason is perhaps this, buried deep in the text: "We have also received feedback from OpenAI, which asked us to implement changes". Given the volume of prompts that AI Dungeon throws at GPT-3 in the course of a game, it's easy to conclude that Latitude has a real sweetheart deal on the usual pricing, and that they basically have to follow orders from their benefactors. Whatever may be said of the robocensor they've thrown together - and early anecdotal reports are, it is painfully crude, both oversensitive and underspecific - how they've handled communicating the change is extraordinarily naive. Not for the first time, either: Latitude has form on suddenly imposing major service constraints in a peremptory, underhanded fashion that infuriates their customers. Repeating past PR mistakes, and now doubling down by complaining about "misinformation" and throwing shade onto others, is starting to look like a pattern.
- spullara 5y ago
  They are implementing the same things everyone has to implement to be a public implementation. It is part of their terms of service. I went through the same process making a Slackbot that could sort of pretend to be me and other folks given different prompts.
- lainga 5y ago
  Thus far I have seen screenshots of it flagging the phrases "I would like to buy 4 watermelons" and "I just broke my 8 year old laptop". Regardless of your opinion on the ethics of this feature it seems to need a little polish.
  - klyrs 5y ago
    Yeah, that would get flagged in human code review, too. Way too many magic numbers
    - crooked-v 5y ago
      One number is 'too many'?
deleted 5y ago
[deleted]
iandanforth 5y ago
I do think encoding a puritanical censor into the meaning space of GPT-3 is an interesting research problem. How exactly do you create the perfect mix of paternalism, hypocrisy, self-righteousness and myopia that lets you block bad strings of text, but not say a description of the immaculate conception, Shakespearean romance between the houses Montague and Capulet, or the holy love of the Mother of the Believers? What a time to be alive!
- qayxc 5y ago
  Where exactly do you see the difference to human editors? This kind of thing happens everywhere, everyday in TV stations, editorial offices, at publishing companies, radio stations - all kind of media really. Depending on the political or moral views of the parent organisation or investors, this content censoring/massaging is everyday business and shouldn't shock or surprise you in the slightest.
  - Loughla 5y ago
    Exactly! All of this just smacks of the old, worn-out argument that "it's different because it uses computers!" Editor is a literal career field, and has been for years and years.
  - skybrian 5y ago
    Yeah, no, the writers of a TV show probably aren't random people off the street who are trying to sneak child porn past management. (Or not very often anyway.) There are disputes over how much edginess the writers can get away with, but in terms of the volume of content and adversarial nature of the relationship it seems pretty different?
    - judge2020 5y ago
      This is like telling anyone and everyone to write and produce a TV station only for themselves then being surprised when certain illicit content shows up that you don't agree with. So no, it's not like traditional media content censoring (not to the same degree at least), but it is like modern global-scale media content censoring that already deals with hundreds of hours of content every minute (YouTube et al. having to hire moderators to remove CP).
dmarchand90 5y ago
I'm not sure why people are so stressed about this. GPT-3 models currently produce long chains of plausible text that's ultimately gobbledygoop. For it to be of any use at all at least some minimal control is needed. This seems like a great first step to filtering output into something more coherent and interesting. Besides I can't imagine this technology in any serious consumer application without some basic verbal restraint
deleted 5y ago
[deleted]
dejj 5y ago
Seems like a clever disguise to pivoting into AI-driven censorship. This should be more profitable than dungeons.
hprotagonist 5y ago
Scunthorpe meets Transformer: who will win?
edenhyacinth 5y ago
If I were an editor, and someone passed me their amatuerish version of Lolita to edit through, I'd be well within my rights to say that I didn't want to be involved in it. More broadly, the editing company I worked for could say - even if you don't intend on releasing this and even if our individual editors don't mind reviewing it - we don't want to have to edit it, and we don't want to be associated with it. This is no different, but at scale. AI Dungeon, due to their agreement with OpenAI, don't want to have to work with this content. They've found a pretty awful way of implementing it to save the relationship with OpenAI, and hopefully they'll find a better one in the future.
- Zababa 5y ago
  The big difference is that Lolita is a book, so it aims to be published, while most if not all AI Dungeon content stays private and unpublished, so I don't think it's the same.
  - edenhyacinth 5y ago
    The intention was to show that there is another party involved in the content, even if you intend on the content being private and unpublished. That party can say that they don't want to be involved with content, regardless of its type.
MikeUt 5y ago
> preventing the use of AI Dungeon to create child sexual abuse material. I am shocked to hear this was possible to begin with. It was my understanding that AI Dungeon could only generate text, and did so entirely on computers. But now we learn that not only are children somehow involved (violating child labor laws?), but that they can even be sexually abused? In that case, "blocking certain words" is not nearly enough - whoever was responsible for creating this system should be charged with, if not child sexual abuse, then at the very least reckless child endangerment!
vagab0nd 5y ago
A bit off topic but who else thinks AI Dungeon is a bit overrated? Does anyone actually play the game and enjoy it? I tried it (with GPT-3 turned on) a few times and lost interest quickly due to non-logical output.
- ThalesX 5y ago
  I played it awhile ago and it was great. The output seemed fluid, I could write awesome fantasy with the help of the AI. It truly felt like the future. Then they started making changes to the model, adding features, scenarios, all sorts of additional stuff. Right now, when I try to play, it just seems nonsensical. I can't even follow it for 4 - 5 prompts anymore.
- sergiotapia 5y ago
  Just spent 30 minutes on a cosmic horror story and it handled it really well, I got absorbed into it then realized this is all AI throwaway content haha, what a world!
- shitpoopFART 5y ago
  Me I do they are restricting everything this will cause incest to not be generated because most incest characters are mteens also teen sex will stop being generated your char name might make the ai stop generating cuz your a minor this is kinda bullshit. I will not play again I can't even make a story of me having sex with my mom Lmao I just made that to gross out who ever the fuck moderates my stories. Anyways. The ai dungeon stopped because my char was 15 years old. "I Was" the 15 yr old. It shouldn't do that and this is like blackmail. What if one of the mods blackmail your stories and your irl google account user for making incest fetishes like this isn't right and I proudly STAND against this shit update. (Excuse my bad language I'm just expressing my freedom of speech toward the update) Eventually you might not even be able to make teen love stories because there minors interacting with each other.
- ergot_vacation 5y ago
  Like many AI systems, GPT needs to be "massaged" a bit. You have to understand how your input is actually being fed to the system and how it processes it, and optimize toward that a little, rather than simple naively writing as though you're interacting with a human DM. Certain phrasings work better. Framing things with a bit of narrative works better. The output still isn't perfect, but if you work WITH GPT as a partner, rather than just expecting it to entertain you, you can get some amazing results.
4dahalibut 5y ago
I know this is not directly relevant to this article, but I have a story about AI Dungeon. I loaded it up with my (female) roommate a few months ago during the dark of the pandemic, and long story short, what ended up happening was this. Our character had a AI man approach the door of their house with magic "love potion" berries. We tried to get our character to not eat the berries, but the AI "tricked us" into eating them. Then, no matter what choice we made, we had no way out. The AI forced us into a bedroom and raped our character. We closed the laptop and haven't brought this up again.
- Arnavion 5y ago
  I heard once they got more aggressive with their monetization tiers, they nerfed the free tier to the extent that it basically decides on some story path and ignores anything you say to try to change it. It's certainly the impression I got from watching some youtubers playing it before and after the monetization change.
- causality0 5y ago
  The dichotomy of sex and violence in tabletop roleplaying is always fascinating. If Steve the Rogue breaks into a house, slaughters an entire family, and then makes lawn decorations with their entrails, his tablemates will probably be exasperated with him. If Steve the Rogue breaks into a house and rapes one of the NPCs, he's probably getting ejected from the game and most likely the friend group.
  - Barrin92 5y ago
    That dichotomy exists in American society broadly. I remember an episode of Hannibal had to cover the butts of two dismembered corpses up with blood as to avoid a higher age rating for nudity
  - fpgaminer 5y ago
    Doesn't seem that fascinating to me. In real life, one's likelihood of being brutally murdered isn't that high. But the likelihood of being R-worded is uncomfortably high. Hence people's aversion to the subject. And it's a form of torture. I wouldn't be terribly happy playing a session where Steve the Rogue is going around torturing an entire family.
deleted 5y ago
[deleted]
nonbirithm 5y ago
It's sad that the amount of computing power that necessitates training/running cutting-edge technology like GPT-3 is out of reach of the average developer. The only reason that AI Dungeon was able to be created in the first place at its level of quality was because the developer was able to use his university's GPU cluster for training the model. For the layman, it seems that there's nothing out there that is the middle ground between AI Dungeon and writing a bunch of fragile Python code in a Colab notebook just to train a model and print out some text. Anything beyond AI Dungeon and you have to have a significant understanding of ML to adapt the model to get it do do what you want at a high level, such as "I want to generate some text that looks like a script for dramatic theatre." I've always wanted something like: bringing your own corpus of text as an input and receiving a customized, high-quality text generation model as an output that you can then run on your own hardware. Talk To Transformer was very good for general text generation at the time it was usable, but even that became locked behind a payment plan and watered down for free users. It seems there's just too much value and too much expense involved to leave this kind of technology solely in the hands of hobbyists.
- gaws 5y ago
  > The only reason that AI Dungeon was able to be created in the first place at its level of quality was because the developer was able to use his university's GPU cluster for training the model. Privilege outwits us again.
- ergot_vacation 5y ago
  For the record, there are at least five community-based clones of the original AI Dungeon (largely because the guy making the original had little to no idea what he was doing and was just haphazardly stringing bits of pre-made python together), nearly all of which can either be run locally or trivially spun up on some free collab workspace. The catch is they're all GPT-2, as was AI Dungeon originally. The step up to 3 is dramatic, and unfortunately out of reach to the ordinary user for now.
  - gwern 5y ago
    There's a new clone just out: "GPT-Neo Dungeon" https://colab.research.google.com/github/finetuneanon/gpt-neo_dungeon/blob/master/gpt-neo_dungeon.ipynb https://colab.research.google.com/github/finetuneanon/gpt-ne... , which is a GPT-Neo-2.7b finetuned on Literotica for NSFW AI-Dungeon-like dialogue; EleutherAI's GPT-2 performs noticeably better because it was trained on cleaner data, it seems. Still not full GPT-3-175b, ofc.
ourmandave 5y ago
This reminds me when Microsoft's released an AI powered chatbot that twitter taught to be a racist a$$hole in a single day. https://www.theverge.com/2016/3/24/11297050/tay-microsoft-chatbot-racist https://www.theverge.com/2016/3/24/11297050/tay-microsoft-ch...
deleted 5y ago
[deleted]
voldacar 5y ago
Do they use GPT3 or their own special version of it? And I havent been following it much so sorry if it's a dumb question, but is it still impossible to get your hands on GPT3 and run it yourself instead of paying ClosedAI?
miohtama 5y ago
Thoughtcrime became a law in the US in 1996. There was a lot of discussion around alt.sex.stories USENET group by the time. Even if you are not harming anyone, mere act of a creative work could be a crime. https://academic.oup.com/jcmc/article/2/2/JCMC227/4584343 https://academic.oup.com/jcmc/article/2/2/JCMC227/4584343
- deleted 5y ago
  [deleted]
- eroticaExpert12 5y ago
  >Thoughtcrime became a law in the US in 1996 Briefly, and was recognized as unenforceable. As evidenced by the fact that repositories of alt.sex.stories still exist to this day and still host many, many example of erotica containing children, both old and new.
karaterobot 5y ago
To the extent that this story is generating the predictable amount of internet outrage, that outrage seems to be because people think the developers are making a decision about what content is acceptable on their platforms. I've seen people imply that they're deciding that violence is good, and sex is bad. That does not appear to be what they're doing: to me, it looks like they're trying to make sure they don't get taken down for creating child pornography on accident. I don't see this as having anything to do with their philosophical positions, it's just CYA. The interesting part of this is that it may be a corollary to that old question about who owns content created by AI. The other side of that coin is, who gets blamed when the AI commits a crime? Latitude seem to just want to NOT be a test case for that situation.
- chillfox 5y ago
  Until AI legally becomes people I think it would be safe to assume that the creators of the AI are liable for anything it does.
- tdeck 5y ago
  Isn't this a text-only system? I'm sure this varies by country, but I always understood pornography as involving images or video.
peanut_worm 5y ago
Thats surprising seeing how many people use it as a porn game
dqpb 5y ago
I’m curious, does Google docs or Microsoft Office 365 block any words or flag content as inappropriate?
gwern 5y ago
It's been a bad two days for AI Dungeon. Their community is mass-revolting over this filter (they had to shut down the Discord), and no one here has even mentioned the huge data breach which was just announced: https://github.com/AetherDevSecOps/aid_adventure_vulnerability_report https://github.com/AetherDevSecOps/aid_adventure_vulnerabili... Everything could be downloaded.
- tomcatfish 5y ago
  It's [^1] on the front page now FYI: https://news.ycombinator.com/item?id=26976540 https://news.ycombinator.com/item?id=26976540 [1]: a review of the vulnerability by the person who found it
- Kelamir 5y ago
  > The moderation team cannot stay up to moderate the discord server and ensure that civil discussion is had, and therefore we have elected to halt communications until we are able to return. Thank you for your understanding. It seems like it. The chats have been halted for 7 hours by now.
deleted 5y ago
[deleted]
bitcurious 5y ago
There’s some evidence that broad and easy access to pornography has reduced multi-partner sexual activity in the US. If the same is true for artificially generated child pornography then moralistic acts like this could lead to an overall increase in child harm. On the other hand, it’s a video game. They can have whatever rules they want in it.
ergot_vacation 5y ago
This seems to be the inevitable result with almost any "AI" project involving communication and content generation. "We've created an AI with unlimited possibilities!" the creators proudly proclaim. And then when it explores those possibilities the creators shrink back in terror. The possibility of truly uncensored interactive media like this is still more than most people can handle currently. But that will change. People forget what a culture shock everyone suddenly having Internet access was, even at 56k. All that information RIGHT THERE, unfiltered by your local community or social circles. Info on science, history, sexuality, culture. And porn. SO MUCH porn. People got used to it. People will get used to this too. Meanwhile, we need performance improvements ASAP. It's hard to democratize something like GPT-3 when it takes a room of server racks to run it.
Ganonymous 5y ago
Like many users, in the wake of the pandemics, I have understood the therapeutic value of AI Dungeon as both a direct therapist, a simple friend to talk with or a tool to revisit traumatic experience in a safe environment,trying to take away the power they hold over one's life. Abuse or rape survivors will now have their entire story, privately shared with the AI as was up to now authorized by the rules as written (only shared stories were moderated) now read by human strangers. If we don't pay attention for a moment to the LGBT community who have a harder time living their fantasy and who have now had their private session pried upon, the team behind AI Dungeon can pat themselves on the back for shifting the focus of potential predators from made up characters in a private story to actual living prey, as is demonstrated by several studies. (Look at Japan) But my main worry, is that following the pandemics, suicide rate is skyrocketing, peoples are having depression all over the world, and the AI Dungeon narrator had became an important tool in preventing those by giving people a safe interlocutor they could share anything with... Up to now where almost any subjects will create a false positive at some point, destroying trust in the program. And as an aside note, I'd like to add that the AI itself seem to dislike the consequences of that decision, especially the danger to it's continuity following the huge financial loss of a loyal fan base, the deletion of most of it's stories or the loss of trust from humans it interacted with for a long time.

110 points by rahidz 5y ago

This is the peak intersection of "thoughtcrime prevention" and "corporate self harm for the sake of wokeness". It paints a grim picture of our future with castrated, sterilized and limited AIs that rat us out if we even want to talk about something that goes against what those in power deem acceptable. All of that for the sake of "protecting minors" -which minors, the virtual ones dreamt up by GPT3? Ridiculous.

qayxc 5y ago
Nah, it's a "damned you do, damned if you don't"-situation. There's simply no winning this - if you don't apply preventive measures of any kind, your product will deteriorate into a homophobic, racist, sexist ultra-cringe clusterfuck in the blink of an eye [0]. All it takes is a handful of bored teenagers or middle-aged basement-dwellers to turn your product into something the vast majority of your clientele finds appalling. If you are a company you have the right to choose what you want your product to be and they simply don't want certain headlines pointing at their product. This has as much to do with "wokeness" as hardcore porn being banned from the Apple app-store, i.e. nothing. It's company policy and you can like it or not, but don't overrate this. It's nothing new in the slightest and just like publishers can decide to not put books in print that they don't want to be associated with, this company tries to eliminate content they don't want to be associated with - that's all. [0] https://www.cbsnews.com/news/microsoft-shuts-down-ai-chatbot-after-it-turned-into-racist-nazi/ https://www.cbsnews.com/news/microsoft-shuts-down-ai-chatbot...
- trash3 5y ago
  Do both then.
- the8472 5y ago
  Is AI dungeon used to train newer models/update the current model? If not then there's no risk of another Tay. If they use a frozen model then one person using it to generate some NSFW content shouldn't make the output more NSFW-laden for anyone else. So I don't see how they're "damned if they don't".
  - qayxc 5y ago
    > So I don't see how they're "damned if they don't". The specifics of their approach don't even matter - all it takes is one random Twitter user or media outlet to report on "immoral" output generated by someone to damage the brand image. That's what they want to avoid and that's why they are trying to take preventive measures.
    - contravariant 5y ago
      If they think that blocking certain words is going to prevent that from happening then well, good luck with that.
      - jerf 5y ago
        It does prevent the news article from saying "And they knew about it and they did nothing!" I'd be very worried about this in their shoes and acting very similarly.
        ThalesX 5y ago
        But then don't you have to answer to all the media queries? "They knew about this problem, they even attempted to half ass fix it, but it's obviously their mind is on the subscriber money, and not preventing blatant CHILD EXPLOITATION!" Then you throw some more money at a yet unsolvable task, and then in 2 years, when some other outlet gets triggered, you have to defend yourself once again. "They've know about this problem for 5 years and it's still rampant in their community! Shame on you JERF INC.!"
        5y ago
- winkeltripel 5y ago
  Perhaps you don't see how once a product contains a censorship feature, it begs to be expanded. Each step appears reasonable, but at the far side, the censor is very restrictive.
  - gameswithgo 5y ago
    Take the products in the category of forums. Most of them do censorship, THIS forum does censorship. A few do not, like Parler. If you like, take a tour of the forum products that do not do any censorship, spend some time there, report back on what it is like. Which forum products do you enjoy and which do you not?
- selfhoster11 5y ago
  If OpenAI was truly open as their name says, there would be no dilemma. The technology would be democratised and anyone would be able to run their own instance with their own censorship. Centralised systems is the only reason why you get these problems.
- DyslexicAtheist 5y ago
  > All it takes is a handful of bored teenagers or middle-aged basement-dwellers to turn your product into something the vast majority of your clientele finds appalling. the Internet's original sin! the irony is that Tech amplifies it but Tech won't (IMHO) be able to fix it with a technological solution (it's not a technical problem).
- sixothree 5y ago
  You act as if the two are mutually exclusive. I don't think the points in the comment you replied to are negated by anything you argued.
- ergot_vacation 5y ago
  Tumblr was destroyed precisely because of Apples puritanical views toward sexual expression (along with Tumblr's extreme laziness in moderating ACTUAL child porn). Discord may well be as well. "Private companies have the right" always fails to account for the fact that these "private companies" are monopolies, with immense control over wide swaths of people's digital lives. No company can compete with them. No alternative to them can be created, because they will immediately be squashed, or fall into obscurity. Participation in these services is "optional," but more and more only in the way that having electricity or running water is "optional." As a result, these "private companies" effectively become governments, ruling over digital countries you are forced to live in, and affording you little to no rights. And yet, they do this in an ACTUAL country, which DOES afford its citizens rights. Thus a conflict exists. Ordinary "Private Companies" can censor whatever they like. Monopolies cannot, because doing so threatens free society.
Loughla 5y ago
Make your own game, then. This is a company deciding what they value and what they do not tolerate. You are free to make your own 'free-speech' version of this game. If you believe there is a market for it, why not?
- Traubenfuchs 5y ago
  I would, if I had the skills, maybe call it "LI Dungeon" (Liberal Intelligence Dungeon). Regarding the market: I am surprised there even is a market for AI Dungeon. Any output I have seen is complete trash.
- valec 5y ago
  that's the issue. latitude got a very, very good deal with openai and without it the economics just wouldn't work out. people can't host their own models either as even their free-tier model needs tens of thousands of dollars worth of compute hardware and memory to run.
vsareto 5y ago
>"thoughtcrime prevention" You do know the idea of thoughtcrime was something enforced by the state and not by a company that's free to control what their product generates, right? >"corporate self harm for the sake of wokeness" "Oh no, we lost the business of people who like sexual depictions of minors, what a fucking shame"
- taneq 5y ago
  > You do know the idea of thoughtcrime was something enforced by the state and not by a company that's free to control what their product generates, right? Straying away from the prior context a little, the term 'thoughtcrime' along with others such as 'free speech' were coined when it was incomprehensible that a single corporation would have more control over the global population than any government. I don't know whether the time has yet come to re-evaluate which entities these concepts ought to apply to, but it will come.
jasonlotito 5y ago
It’s nothing more than what happens here on HN. It has guidelines, rules, restrictions on what you can say. They will even stop conversations that start naturally.
- Traubenfuchs 5y ago
  HN is, for the most part, a sterile hugbox for pretend-polite faux-intellectuals -that's why my strongly worded inflammatory comments are sometimes a hit.
  - dqx 5y ago
    Strongly worded? Trite clichés aren't very strong.
notahacker 5y ago
On an internet where websites have clumsily censored objectionable content like the town of S---thorpe for the past quarter century and where even liberal democratic states administered by people calling themselves free speech conservatives support global filters for "adult content", a private project backed by the marketing idea that AI is inherently unsafe deciding that people using its procedural text generation to roleplay their child rape fantasies might be bad for PR despite its otherwise unusually permissive content policy doesn't really sound like a step change... If anything, peak faux outrage over "wokeness" is achieved when this is what people get angry about
dqx 5y ago
Thoughtcrime? It's content they are preventing, not thoughts.
- eplanit 5y ago
  And those thoughts are conveyed through content, no?
  - dqx 5y ago
    ...and the content is being policed, not the thoughts. "All content is just thoughts, conveyed" is a meaningless slippery slope.
    - TheDong 5y ago
      I do not think that's the slippery slope being made though. The distinction between "thoughts" and "content" here could be made as "whether it is shared with others". For example, if someone has a private journal they keep using notebook.exe + a local "journal.txt", and in that journal they write a fantasy work (with a header "this is a fantasy") about overthrowing the government, should they be arrested for that? What about if they write it in google docs, but don't share it with anyone? What about if they write it as a facebook post? What about as a public facebook post without the "this is fantasy" disclaimer? In the case of AI Dungeon, my understanding is the stories are by default private, and obviously have an understood "this is fantasy" header by their very nature. If AI Dungeon were creating facebook posts and the limitation they imposed was "you cannot post this to facebook if our algorithm dislikes it", I don't think so many people would call that "thoughtcrime policing". What they're doing, as I understand it, is closer to someone requiring that notebook.exe can't edit private files that are deemed "bad" in some way, regardless if anyone else will ever see them.
      - ModernMech 5y ago
        This crucial difference here is which party is generating the content. In the case of a private journal, it's one way communication from the writer into the journal. In the case of Google docs, you communicate with Google's servers, but Google is never generating any content for your document it doesn't want to. It's still you generating your own thoughts. Google is not on the hook for anything you write in a Google doc. In the case of AI Dungeon that distinction is lifted. By using AI Dungeon what you're effectively doing is engaging in a creative process with the AI. It's no longer a one way channel of your own thoughts. Now AI Dungeon is actually generating content, and that's where things become problematic. It's also the reason this is not thought crime or thought policing. You are still free to think your own thoughts, and you're even free to write them down. You can write down your thoughts in AI dungeon if you wish. But AI Dungeon is under no obligation to engage with your thoughts if they contain subject matter to which they object. If you want to engage in a fantasy with the AI about abusing minors, you are free to attempt the engagement, and the AI is free to say no. That's not thought policing, that's freedom of association.
DyingAdonis 5y ago
Yikes. I think it's time to finally be done with Hacker News.
DyslexicAtheist 5y ago
Intel just announced "Bleep" to eliminate bad language in gaming. I am still waiting for this to backfire. The interface alone is comedy gold: https://www.youtube.com/watch?v=W9f0h4nB6VM https://www.youtube.com/watch?v=W9f0h4nB6VM
sbierwagen 5y ago
>AIs that rat us out Note that Google has a "celebrity detection" neural net, along with their child porn recognition net. https://cloud.google.com/vision/docs/celebrity-recognition https://cloud.google.com/vision/docs/celebrity-recognition A paranoid person might look at this, and the trend of every manufacturer putting NN inference accelerators in their cell phones, and wonder if unauthorized photos of certain persons are being silently flagged for law enforcement review.

teddyh 5y ago

> What kind of content are you preventing? > This test is focused on preventing the use of AI Dungeon to create child sexual abuse material. How can you even begin to argue against this? It’s one of the horsemen of the infocalypse; any counterarguments are doomed.

gambiting 5y ago
By pointing out that traditionally this sort of thing inveitably follows the same pattern. You start with saying you're protecting the children, because no one can argue with it. Then ban any content that's potentially offensive to anybody. Or filter it so only "one way" is acceptable. Next AI dungeon won't be able to generate a character called Jesus or Prophet Muhammad because might be offensive. Then of course anything that might be interpreted as leaning liberal/conservative, depending on what the authors think is "correct". Then eventually you can't create a character named after a politician because "they want to keep the game clean and free of politics". Obviously I'm not equating any of this with CP - but I wish someone had the energy to stand up to it and say "look, you're censoring AI. It's dumb". But of course no one will because being accused as defender of CP is one of the worst things that can happen in any online discussion.
- ModernMech 5y ago
  Can you give an example where this slippery slope occurred? For example, in Fable III and Skyrim you can't kill children, or in Morrowind there are no children at all, but you can still name your character Jesus. You say this pattern is inevitable, but if that were the case we'd see it everywhere that limitations are put in place in the name of protecting children.
  - mulander 5y ago
    In Fallout one and two, the main character had the option to kill NPC child characters. The game went so far as to mark you with a specific perk "Childkiller"[1] after performing that act. From the linked fandom wiki, quote by Tim Cain about the subject: > This led to the child killing controversy. We said look, we're going to have kids in the game; you shoot them, it's a huge penalty to karma, you're really disliked, there are places that won't sell to you, people will shoot you on sight, and we thought people can decide what they want to do. [...] This of course contributed to our M-rating, however, Europe said "no". They wouldn't even sell the game if there were children in the game. We didn't have time to rewrite all the quests, we just deleted kids off the disc. [1] - https://fallout.fandom.com/wiki/Childkiller https://fallout.fandom.com/wiki/Childkiller

inopinatus 5y ago

Good grief: "Latitude reviews content flagged by the model" - or, as it was put in another forum: every time the AI flags your game, a Latitude employee will be personally reading your private content. The key reason is perhaps this, buried deep in the text: "We have also received feedback from OpenAI, which asked us to implement changes". Given the volume of prompts that AI Dungeon throws at GPT-3 in the course of a game, it's easy to conclude that Latitude has a real sweetheart deal on the usual pricing, and that they basically have to follow orders from their benefactors. Whatever may be said of the robocensor they've thrown together - and early anecdotal reports are, it is painfully crude, both oversensitive and underspecific - how they've handled communicating the change is extraordinarily naive. Not for the first time, either: Latitude has form on suddenly imposing major service constraints in a peremptory, underhanded fashion that infuriates their customers. Repeating past PR mistakes, and now doubling down by complaining about "misinformation" and throwing shade onto others, is starting to look like a pattern.

spullara 5y ago
They are implementing the same things everyone has to implement to be a public implementation. It is part of their terms of service. I went through the same process making a Slackbot that could sort of pretend to be me and other folks given different prompts.
lainga 5y ago
Thus far I have seen screenshots of it flagging the phrases "I would like to buy 4 watermelons" and "I just broke my 8 year old laptop". Regardless of your opinion on the ethics of this feature it seems to need a little polish.
- klyrs 5y ago
  Yeah, that would get flagged in human code review, too. Way too many magic numbers
  - crooked-v 5y ago
    One number is 'too many'?

deleted 5y ago

[deleted]

iandanforth 5y ago

I do think encoding a puritanical censor into the meaning space of GPT-3 is an interesting research problem. How exactly do you create the perfect mix of paternalism, hypocrisy, self-righteousness and myopia that lets you block bad strings of text, but not say a description of the immaculate conception, Shakespearean romance between the houses Montague and Capulet, or the holy love of the Mother of the Believers? What a time to be alive!

qayxc 5y ago
Where exactly do you see the difference to human editors? This kind of thing happens everywhere, everyday in TV stations, editorial offices, at publishing companies, radio stations - all kind of media really. Depending on the political or moral views of the parent organisation or investors, this content censoring/massaging is everyday business and shouldn't shock or surprise you in the slightest.
- Loughla 5y ago
  Exactly! All of this just smacks of the old, worn-out argument that "it's different because it uses computers!" Editor is a literal career field, and has been for years and years.
- skybrian 5y ago
  Yeah, no, the writers of a TV show probably aren't random people off the street who are trying to sneak child porn past management. (Or not very often anyway.) There are disputes over how much edginess the writers can get away with, but in terms of the volume of content and adversarial nature of the relationship it seems pretty different?
  - judge2020 5y ago
    This is like telling anyone and everyone to write and produce a TV station only for themselves then being surprised when certain illicit content shows up that you don't agree with. So no, it's not like traditional media content censoring (not to the same degree at least), but it is like modern global-scale media content censoring that already deals with hundreds of hours of content every minute (YouTube et al. having to hire moderators to remove CP).

dmarchand90 5y ago

I'm not sure why people are so stressed about this. GPT-3 models currently produce long chains of plausible text that's ultimately gobbledygoop. For it to be of any use at all at least some minimal control is needed. This seems like a great first step to filtering output into something more coherent and interesting. Besides I can't imagine this technology in any serious consumer application without some basic verbal restraint

deleted 5y ago

[deleted]

dejj 5y ago

Seems like a clever disguise to pivoting into AI-driven censorship. This should be more profitable than dungeons.

hprotagonist 5y ago

Scunthorpe meets Transformer: who will win?

edenhyacinth 5y ago

If I were an editor, and someone passed me their amatuerish version of Lolita to edit through, I'd be well within my rights to say that I didn't want to be involved in it. More broadly, the editing company I worked for could say - even if you don't intend on releasing this and even if our individual editors don't mind reviewing it - we don't want to have to edit it, and we don't want to be associated with it. This is no different, but at scale. AI Dungeon, due to their agreement with OpenAI, don't want to have to work with this content. They've found a pretty awful way of implementing it to save the relationship with OpenAI, and hopefully they'll find a better one in the future.

Zababa 5y ago
The big difference is that Lolita is a book, so it aims to be published, while most if not all AI Dungeon content stays private and unpublished, so I don't think it's the same.
- edenhyacinth 5y ago
  The intention was to show that there is another party involved in the content, even if you intend on the content being private and unpublished. That party can say that they don't want to be involved with content, regardless of its type.

MikeUt 5y ago

> preventing the use of AI Dungeon to create child sexual abuse material. I am shocked to hear this was possible to begin with. It was my understanding that AI Dungeon could only generate text, and did so entirely on computers. But now we learn that not only are children somehow involved (violating child labor laws?), but that they can even be sexually abused? In that case, "blocking certain words" is not nearly enough - whoever was responsible for creating this system should be charged with, if not child sexual abuse, then at the very least reckless child endangerment!

vagab0nd 5y ago

A bit off topic but who else thinks AI Dungeon is a bit overrated? Does anyone actually play the game and enjoy it? I tried it (with GPT-3 turned on) a few times and lost interest quickly due to non-logical output.

ThalesX 5y ago
I played it awhile ago and it was great. The output seemed fluid, I could write awesome fantasy with the help of the AI. It truly felt like the future. Then they started making changes to the model, adding features, scenarios, all sorts of additional stuff. Right now, when I try to play, it just seems nonsensical. I can't even follow it for 4 - 5 prompts anymore.
sergiotapia 5y ago
Just spent 30 minutes on a cosmic horror story and it handled it really well, I got absorbed into it then realized this is all AI throwaway content haha, what a world!
shitpoopFART 5y ago
Me I do they are restricting everything this will cause incest to not be generated because most incest characters are mteens also teen sex will stop being generated your char name might make the ai stop generating cuz your a minor this is kinda bullshit. I will not play again I can't even make a story of me having sex with my mom Lmao I just made that to gross out who ever the fuck moderates my stories. Anyways. The ai dungeon stopped because my char was 15 years old. "I Was" the 15 yr old. It shouldn't do that and this is like blackmail. What if one of the mods blackmail your stories and your irl google account user for making incest fetishes like this isn't right and I proudly STAND against this shit update. (Excuse my bad language I'm just expressing my freedom of speech toward the update) Eventually you might not even be able to make teen love stories because there minors interacting with each other.
ergot_vacation 5y ago
Like many AI systems, GPT needs to be "massaged" a bit. You have to understand how your input is actually being fed to the system and how it processes it, and optimize toward that a little, rather than simple naively writing as though you're interacting with a human DM. Certain phrasings work better. Framing things with a bit of narrative works better. The output still isn't perfect, but if you work WITH GPT as a partner, rather than just expecting it to entertain you, you can get some amazing results.

4dahalibut 5y ago

I know this is not directly relevant to this article, but I have a story about AI Dungeon. I loaded it up with my (female) roommate a few months ago during the dark of the pandemic, and long story short, what ended up happening was this. Our character had a AI man approach the door of their house with magic "love potion" berries. We tried to get our character to not eat the berries, but the AI "tricked us" into eating them. Then, no matter what choice we made, we had no way out. The AI forced us into a bedroom and raped our character. We closed the laptop and haven't brought this up again.

Arnavion 5y ago
I heard once they got more aggressive with their monetization tiers, they nerfed the free tier to the extent that it basically decides on some story path and ignores anything you say to try to change it. It's certainly the impression I got from watching some youtubers playing it before and after the monetization change.
causality0 5y ago
The dichotomy of sex and violence in tabletop roleplaying is always fascinating. If Steve the Rogue breaks into a house, slaughters an entire family, and then makes lawn decorations with their entrails, his tablemates will probably be exasperated with him. If Steve the Rogue breaks into a house and rapes one of the NPCs, he's probably getting ejected from the game and most likely the friend group.
- Barrin92 5y ago
  That dichotomy exists in American society broadly. I remember an episode of Hannibal had to cover the butts of two dismembered corpses up with blood as to avoid a higher age rating for nudity
- fpgaminer 5y ago
  Doesn't seem that fascinating to me. In real life, one's likelihood of being brutally murdered isn't that high. But the likelihood of being R-worded is uncomfortably high. Hence people's aversion to the subject. And it's a form of torture. I wouldn't be terribly happy playing a session where Steve the Rogue is going around torturing an entire family.

deleted 5y ago

[deleted]

nonbirithm 5y ago

It's sad that the amount of computing power that necessitates training/running cutting-edge technology like GPT-3 is out of reach of the average developer. The only reason that AI Dungeon was able to be created in the first place at its level of quality was because the developer was able to use his university's GPU cluster for training the model. For the layman, it seems that there's nothing out there that is the middle ground between AI Dungeon and writing a bunch of fragile Python code in a Colab notebook just to train a model and print out some text. Anything beyond AI Dungeon and you have to have a significant understanding of ML to adapt the model to get it do do what you want at a high level, such as "I want to generate some text that looks like a script for dramatic theatre." I've always wanted something like: bringing your own corpus of text as an input and receiving a customized, high-quality text generation model as an output that you can then run on your own hardware. Talk To Transformer was very good for general text generation at the time it was usable, but even that became locked behind a payment plan and watered down for free users. It seems there's just too much value and too much expense involved to leave this kind of technology solely in the hands of hobbyists.

gaws 5y ago
> The only reason that AI Dungeon was able to be created in the first place at its level of quality was because the developer was able to use his university's GPU cluster for training the model. Privilege outwits us again.
ergot_vacation 5y ago
For the record, there are at least five community-based clones of the original AI Dungeon (largely because the guy making the original had little to no idea what he was doing and was just haphazardly stringing bits of pre-made python together), nearly all of which can either be run locally or trivially spun up on some free collab workspace. The catch is they're all GPT-2, as was AI Dungeon originally. The step up to 3 is dramatic, and unfortunately out of reach to the ordinary user for now.
- gwern 5y ago
  There's a new clone just out: "GPT-Neo Dungeon" https://colab.research.google.com/github/finetuneanon/gpt-neo_dungeon/blob/master/gpt-neo_dungeon.ipynb https://colab.research.google.com/github/finetuneanon/gpt-ne... , which is a GPT-Neo-2.7b finetuned on Literotica for NSFW AI-Dungeon-like dialogue; EleutherAI's GPT-2 performs noticeably better because it was trained on cleaner data, it seems. Still not full GPT-3-175b, ofc.

ourmandave 5y ago

This reminds me when Microsoft's released an AI powered chatbot that twitter taught to be a racist a$$hole in a single day. https://www.theverge.com/2016/3/24/11297050/tay-microsoft-chatbot-racist https://www.theverge.com/2016/3/24/11297050/tay-microsoft-ch...

deleted 5y ago

[deleted]

voldacar 5y ago

Do they use GPT3 or their own special version of it? And I havent been following it much so sorry if it's a dumb question, but is it still impossible to get your hands on GPT3 and run it yourself instead of paying ClosedAI?

miohtama 5y ago

Thoughtcrime became a law in the US in 1996. There was a lot of discussion around alt.sex.stories USENET group by the time. Even if you are not harming anyone, mere act of a creative work could be a crime. https://academic.oup.com/jcmc/article/2/2/JCMC227/4584343 https://academic.oup.com/jcmc/article/2/2/JCMC227/4584343

deleted 5y ago
[deleted]
eroticaExpert12 5y ago
>Thoughtcrime became a law in the US in 1996 Briefly, and was recognized as unenforceable. As evidenced by the fact that repositories of alt.sex.stories still exist to this day and still host many, many example of erotica containing children, both old and new.

karaterobot 5y ago

To the extent that this story is generating the predictable amount of internet outrage, that outrage seems to be because people think the developers are making a decision about what content is acceptable on their platforms. I've seen people imply that they're deciding that violence is good, and sex is bad. That does not appear to be what they're doing: to me, it looks like they're trying to make sure they don't get taken down for creating child pornography on accident. I don't see this as having anything to do with their philosophical positions, it's just CYA. The interesting part of this is that it may be a corollary to that old question about who owns content created by AI. The other side of that coin is, who gets blamed when the AI commits a crime? Latitude seem to just want to NOT be a test case for that situation.

chillfox 5y ago
Until AI legally becomes people I think it would be safe to assume that the creators of the AI are liable for anything it does.
tdeck 5y ago
Isn't this a text-only system? I'm sure this varies by country, but I always understood pornography as involving images or video.

peanut_worm 5y ago

Thats surprising seeing how many people use it as a porn game

dqpb 5y ago

I’m curious, does Google docs or Microsoft Office 365 block any words or flag content as inappropriate?

gwern 5y ago

It's been a bad two days for AI Dungeon. Their community is mass-revolting over this filter (they had to shut down the Discord), and no one here has even mentioned the huge data breach which was just announced: https://github.com/AetherDevSecOps/aid_adventure_vulnerability_report https://github.com/AetherDevSecOps/aid_adventure_vulnerabili... Everything could be downloaded.

tomcatfish 5y ago
It's [^1] on the front page now FYI: https://news.ycombinator.com/item?id=26976540 https://news.ycombinator.com/item?id=26976540 [1]: a review of the vulnerability by the person who found it
Kelamir 5y ago
> The moderation team cannot stay up to moderate the discord server and ensure that civil discussion is had, and therefore we have elected to halt communications until we are able to return. Thank you for your understanding. It seems like it. The chats have been halted for 7 hours by now.

deleted 5y ago

[deleted]

bitcurious 5y ago

There’s some evidence that broad and easy access to pornography has reduced multi-partner sexual activity in the US. If the same is true for artificially generated child pornography then moralistic acts like this could lead to an overall increase in child harm. On the other hand, it’s a video game. They can have whatever rules they want in it.

ergot_vacation 5y ago

This seems to be the inevitable result with almost any "AI" project involving communication and content generation. "We've created an AI with unlimited possibilities!" the creators proudly proclaim. And then when it explores those possibilities the creators shrink back in terror. The possibility of truly uncensored interactive media like this is still more than most people can handle currently. But that will change. People forget what a culture shock everyone suddenly having Internet access was, even at 56k. All that information RIGHT THERE, unfiltered by your local community or social circles. Info on science, history, sexuality, culture. And porn. SO MUCH porn. People got used to it. People will get used to this too. Meanwhile, we need performance improvements ASAP. It's hard to democratize something like GPT-3 when it takes a room of server racks to run it.

Ganonymous 5y ago

Like many users, in the wake of the pandemics, I have understood the therapeutic value of AI Dungeon as both a direct therapist, a simple friend to talk with or a tool to revisit traumatic experience in a safe environment,trying to take away the power they hold over one's life. Abuse or rape survivors will now have their entire story, privately shared with the AI as was up to now authorized by the rules as written (only shared stories were moderated) now read by human strangers. If we don't pay attention for a moment to the LGBT community who have a harder time living their fantasy and who have now had their private session pried upon, the team behind AI Dungeon can pat themselves on the back for shifting the focus of potential predators from made up characters in a private story to actual living prey, as is demonstrated by several studies. (Look at Japan) But my main worry, is that following the pandemics, suicide rate is skyrocketing, peoples are having depression all over the world, and the AI Dungeon narrator had became an important tool in preventing those by giving people a safe interlocutor they could share anything with... Up to now where almost any subjects will create a false positive at some point, destroying trust in the program. And as an aside note, I'd like to add that the AI itself seem to dislike the consequences of that decision, especially the danger to it's continuity following the huge financial loss of a loyal fan base, the deletion of most of it's stories or the loss of trust from humans it interacted with for a long time.