6 ms·
Character.ai to bar children under 18 from using its chatbots
https://archive.ph/fhVEV https://archive.ph/fhVEV
- ChrisArchitect 1y agoRelated: Teen in love with chatbot killed himself – can the chatbot be held responsible? https://news.ycombinator.com/item?id=45726556 https://news.ycombinator.com/item?id=45726556
- nopurpose 1y agoHow is it possible to develop emotional connection to anyone with a goldfish memory? I'd be surprised if context size there is more than 50K tokens.
- sosodev 1y agoMost AI companions have memory systems. It's still quite simplistic but it's not "goldfish memory" or just limited to the context window. From what I understand, some inputs from the user will trigger a tool call that searches the memory database and other times it will search for a memory randomly. With that said, I think people started falling in love with LLMs before memory systems and they probably even fell in love with chatbots before LLMs. I believe that the simple, unfortunate truth is that love is not at all a rational processes and your body doesn't need much input to produce those feelings.
- ambicapter 1y agoPeople got attached to a bot that only asked questions.
- hackernewds 1y agoPeople are lonely and desire connection.
- PlunderBunny 1y agoPeople were developing an emotional connection to the Eliza program in the late 1960s [0], the code for which would have contained less characters than the discussion on this page. 0. https://www.theguardian.com/technology/2023/jul/25/joseph-weizenbaum-inventor-eliza-chatbot-turned-against-artificial-intelligence-ai https://www.theguardian.com/technology/2023/jul/25/joseph-we...
- goopypoop 1y agoa woman married a bridge in 2013
- hatefulheart 1y agoIt makes no difference to their bottom line. After all, appealing to children over the age of 18 is where LLMs find their market.
- hackernewds 1y agoIf they are banning a large part of their current and future market, with competitors serving the space, how does it not affect their bottom line?
- BoorishBears 1y agoMost competitors can't successfully serve kids yet. These kids hammer H100s for 30+ hours a week but will revolt at ads or the idea of paying money. C.ai probably only exists at its current size because Noam had access cheap access to TPUs and people who can scale inference on them at the earliest stages of their growth (and obviously because he could raise with his pedigree, but looking at things others deal with) Eventually if the unit economics start to work they can always roll this back, but I think people are underestimating how much of a positive this is for them
- red2awn 1y agoThey are not a profitable company at all. They only started monetizing this year and the customer base is not a fan of it at all.
- thw_9a83c 1y ago"Going forward, it will use technology to detect underage users based on conversations and interactions on the platform, as well as information from any connected social media accounts" Something tells me that this ain't gonna work. Kids and teens are more inventive than they probably realize.
- add-sub-mul-div 1y agoA model would also have to account for the emotionally stunted adults using an AI bf/gf service.
- gs17 1y agoI doubt they care that much, since the cost of a false positive is that user having to verify their age and the alternative is all users having to verify their age.
- BoorishBears 1y agoThe number 1 reason they're doing this is so they can loosen the clamps on NSFW/romance, and they've previously released models targeted at adults intended to improve performance at that.
- gs17 1y agoEspecially now that OpenAI is entering that space officially by the end of the year.
- gs17 1y agoI don't think the "use technology to detect underage users" approach has ever worked well for its stated intent (it works okay for being able to say you're "doing something" about the problem while not losing as many users), but it's slightly better than mandatory ID for everyone.
- mothballed 1y agoIt's just a way to mitigate being sued when inevitably the lawsuits pour in that the AI-assisted harmful decisions that were made totally weren't shared fault of the parents, other external environment, and/or unlucky genetics.
- causal 1y agoFinally. CAI is notoriously shady with harvesting, selling, and making it difficult to remove data. How many kids have CAI's chatbots already seduced and had their intimate conversations sold to Google?
- BoorishBears 1y agoI suspect Google is the one pushing for this. C.ai has dramatically folded on two of its longest standing struggles in the last two months: underage users and intellectual property In both cases they went nuclear in a way that implies they actually don't care if the current product survives as long as C.ai (read: Google) isn't exposed to the ongoing risk
- TZubiri 1y agoVery nice. Just yesterday I wrote about the 13-18 age group using ChatGPT and how I think it should be disallowed (without guardian consent), this was in the context of suicide cases. https://news.ycombinator.com/item?id=45733618 https://news.ycombinator.com/item?id=45733618 On a similar note, I was completing my application for YC Startup School / Co-Founder matching program. And when listing possible ideas for startups I straight out explicitly mentioned I'm not interested in pursuing AI ideas at the moment, AI features are fine, but not as the main pitch. It feels like at least for me the bubble has popped, I have talked also recently about the way in which the bubble might pop would be due to legal liability collapse in the courts. https://news.ycombinator.com/item?id=45727060 https://news.ycombinator.com/item?id=45727060 This added with the fact that AI was always a vague folk category of software, it's being used for robotics, NLP and fake images, I just don't think it's a real taxon. Similar to the crypto buzz from the last season, the reputable parties will exit and stop associating, while the grifters and free-associating mercenaries will remain. Even if you are completely selfish, it's not even hugely beneficial to be in the "AI" space, at least in my experience, customers come in with huge expectations, and non-huge budgets. Even if you sell your soul to implement a chatbot that will replace 911 operators, at this point the major actors have already done so, or not, and you are left with small companies that want to be able to fire 5 employees and will pay you 3 months of employee salary if you can get it done by vibe code completing their vibe coded prototype within a 2-3 deadline.
- iteria 1y agoIf parental consent is all that's necessary then you might as well blanket allow it. I'm a parent and interacting with a lot of parents of young children, the vast majority of them can't be bothered to deal with whining or inconvenience. Which is why I see 8 year olds with cash app even though that's not allowed any many abandon or extremely weaken parental controls on basically every advice. My personal favorite story is when I talk my youngest aunt about a videogame my cousin wanted. She said no absolutely not. Then proceeded to buy the game for ehr 10 year old. A game she was carded for. A game that has that it's M rated and has adult themes and whatnot on the box. She called me later in horror about how inappropriate this game I told her about 2 weeks earlier was. How could they make games like that for children she says about the game she was carded for because it's only for adults. I use her as an example but that situation is a lot of parents. I personally think that it's not the government's place to say how much exposure I want to give my child to the internet, but I have rules and boundaries around that with my kid. Many of her friends have free access and and have always had it since toddlers. People say it's parents not being savvy, but honestly it's parents not caring. Parent controls have been around over 30 years and they have always been dead simple. But they do increase the whining in your life from your kids and that means if parents can allow it a high quantity will. I have no faith that a law will stop significantly more kids than no law. I know too many parents who allow their kids to do things they know are harmful to their kid because "I don't want them to feel left out" or they don't want to deal with whining.
- ivape 1y agoThis is the only way. Tech companies cannot become like the police in many countries. The police in many countries are used as a "catch-all" function at the moment of, where they have to deal with the failure of other functions (parenting, community) exactly at the moment of (just-in time catch-all). Your kid is on the fucking computer all day building an unhealthy relationship with essentially a computer game character. Step the fuck in. These companies absolutely have to make liability clear here. It's an 18+ product, watch your kids.
- throwaway314155 1y ago> This is the only way. You're more optimistic than I am. Their announcement is virtue signaling at best. Nothing will come from this. Kids will figure out a way around their so-called detection mechanisms because if there were any false positives for adults they would lose adult customers _and_ kids customers.
- ivape 1y agoNothing can be stopped. All that can be done is make clear where everyone stands. We're very early in all of this, and it's important to make clear what liability is. You want to sail out to the new world with everyone else, accept the terms, which includes death. You cannot just let your kid touch AI without both eyes open. The universe of " I didn't know the kids would make deep fakes of their classmates ", is yet to come. Some parents going straight to fucking jail. Talk to your kids, things are moving at a dangerous pace.
- wagwang 1y agoStep in as parents or step in as the unabomber?
- alyxya 1y agoI worry that all this does is reduce future liability issues and make those users use another chatbot instead. I trust character.ai more than other chatbots for safety. > [Dr. Nina Vasan] said the company should work with child psychologists and psychiatrists to understand how suddenly losing access to A.I. companions would affect young users. I hope there’s a more gradual transition here for those users. AI companions are often far more available than other people, so it’s easy to talk more and get more attached to them. This restriction may end up being a net negative to affected users.
- Edmond 1y agoThere is a correct way to do age verification (and information verification in general) that supports strong privacy and makes it difficult to evade: https://news.ycombinator.com/item?id=44723418 https://news.ycombinator.com/item?id=44723418 It is also highly compatible with the internet both in terms of technical/performance scalability and utility scalability (you can use it for just about any information verification need in any kind of application).
- Philpax 1y agoUndisclosed self-promotion.
- Edmond 1y agoMy motivation is less about self-promotion at this point and perhaps just frustration with the face-palm quality of the failure to properly implement information verification on the internet. Every time I hear about some dumb approach to age verification (conversation analysis...really?) or a romance scam story because of a fraudster somewhere in Malaysia..I have the need to scream...THERE IS A CORRECT SOLUTION.
- triceratops 1y agoAge verification doesn't have to be perfect or even cryptographically secure. We don't demand it for alcohol or tobacco: carcinogenic, addictive substances that cause (in the case of alcohol) impaired judgment leading to deadly accidents. There's no justification for online age verification to be more invasive or stringent than what's done today for buying alcohol or tobacco IRL. My proposal is here: https://news.ycombinator.com/item?id=45141744 https://news.ycombinator.com/item?id=45141744
- gooseWithFood20 1y agoHonestly this needs to be standardised.
- BatteryMountain 1y agoHow about we just enact a law that state children aren't allowed online, not allowed to interact with interative software (like ai, user generated content). Make the parents responsible, not companies.
- tonyhart7 1y ago"How about we just enact a law that state children aren't allowed online" I literally circumvent website blocking using VPN as a kid, no one can stop anyone from going "online" in 2025
- r0fl 1y agoHow will you enforce this? What if a child is at school where there are Chromebooks and teachers aren’t as tech savvy as the majority of hacker news? What if a child is at a library that has Chromebooks you can take out and use for homework? Wha if a child is at an older cousins place? What if a child is a park with their friends and uses a device? Should parents be next to their child helicopters parenting 24/7? Is that how you remember your childhood? Is that how you currently parent giving zero atonony to children? Blaming parents is ridiculous. Lot of parents aren’t tech savvy and are too dumb to be tech savvy and stay on top of the latest tech thing
- fluoridation 1y agoYou don't enforce it. The point of such laws is not to actually be enforced, but as a legal tool when problems come up. For example, if you as an adult say something inappropriate to a child online thinking it's an adult, you'd be protected if it's ever brought up, because the child shouldn't have been online to begin with and it was not your responsibility to check the other person's age. It's not unlike being in a bar and assuming everyone is 18 or older, because it's the bar's responsibility to forbid entry to anyone younger.
- molave 1y agoSuch a mindset is ripe for selective enforcement and discrimination if society or someone powerful does not like you.
- Sparkyte 1y agoI think ChatBots are not very helpful if you don't treat them with scrutiny. There should be a mandatory requirement of a disclaimer before interacting with all AI.
- amiga386 1y agoThey should ban the over-18s too. https://www.bbc.co.uk/news/technology-67012224 https://www.bbc.co.uk/news/technology-67012224
- harrisonjackson 1y agoLots of threads in here saying this is just a "legal" protection move... I'd like to believe that most actual people want to protect kids. It's easy to write off corporations and forget that they are founded by real people and employ real people... some with kids of their own or with nieces or nephews etc, and some of them probably do really care. Not saying character.ai is driven by that but I imagine the times they've been in the news were genuinely hard times to be working there...
- DuperPower 1y agolol they business idea itself is very easy to degenerate for money so if Who created It didnt thought putting barriers to minors from the beggining the person is stupid or greedy
- b00ty4breakfast 1y agoI'm sure some of them care genuinly but I've interacted with corporations enough to know that the individual proclivities of the parts don't necessarily appear in the whole.
- SoftTalker 1y agoYeah they care so much they strictly limit what their own kids do with the technology. But push it on everyone else.
- ml-anon 1y agoThey (the leadership and the folks who left to Google) absolutely don’t give a shit about protecting kids. In fact most of America doesn’t give a shit about protecting kids (guns, SNAP, vaccines, the goddamn Epstein files).
- rsynnott 1y agoI mean, I think you're probably giving companies a _bit_ too much benefit of the doubt, on this particular issue. Here's another major chatbot peddler: https://www.theguardian.com/technology/2025/aug/15/meta-ai-chat-children https://www.theguardian.com/technology/2025/aug/15/meta-ai-c... > An internal Meta policy document, seen by Reuters, showed the social media giant’s guidelines for its chatbots allowed the AI to “engage a child in conversations that are romantic or sensual”, generate false medical information, and assist users in arguing that Black people are “dumber than white people”
- empath75 1y agoThere is nothing about character.ai that would be appealing for more than 30 minutes to anybody that isn't suffering from some kind of acute mental health crisis. They should shut the whole thing down. That is a cool demo that should have never been made into a product.
- moogly 1y agoI thought that was like 75% of their user base. What do adults use this for?
- x187463 1y agoI'm interested in that question as well. Perhaps there is an adult user here on HN who can describe their use-case.
- Mr_Bees69 1y agoCharacter.ai to go bankrupt.