6 ms·

"Going forward, it will use technology to detect underage users based on conversations and interactions on the platform, as well as information from any connect

"Going forward, it will use technology to detect underage users based on conversations and interactions on the platform, as well as information from any connected social media accounts" Something tells me that this ain't gonna work. Kids and teens are more inventive than they probably realize.

add-sub-mul-div 1y ago
A model would also have to account for the emotionally stunted adults using an AI bf/gf service.
- gs17 1y ago
  I doubt they care that much, since the cost of a false positive is that user having to verify their age and the alternative is all users having to verify their age.
- BoorishBears 1y ago
  The number 1 reason they're doing this is so they can loosen the clamps on NSFW/romance, and they've previously released models targeted at adults intended to improve performance at that.
  - gs17 1y ago
    Especially now that OpenAI is entering that space officially by the end of the year.
gs17 1y ago
I don't think the "use technology to detect underage users" approach has ever worked well for its stated intent (it works okay for being able to say you're "doing something" about the problem while not losing as many users), but it's slightly better than mandatory ID for everyone.
- mothballed 1y ago
  It's just a way to mitigate being sued when inevitably the lawsuits pour in that the AI-assisted harmful decisions that were made totally weren't shared fault of the parents, other external environment, and/or unlucky genetics.
  - wat10000 1y ago
    Right, the stated intent may be to block underage users, but the actual intent is to Do Something so that you can't be accused of Not Doing Something.
  - SoftTalker 1y ago
    And certainly not the fault of the always conscientious tech companies who are actually the ones running the services.
- moi2388 1y ago
  Mandatory ID wouldn’t be that bad if implemented claims-based, from within the browser. So that the browser could ask: Hey government, is this user above 18? And the government just responds with yes or no. Sites would then only need to whitelist browsers with this check an merely see an “allowed” claim for whatever type of site they have. Parents or users could use parental controls to “allow” or “block” certain categories, meaning website creators wouldn’t even get any info on the user.
BoorishBears 1y ago
They don't care: if they put in robust sounding guards and openly state "chatbots are not for kids", they can point to them in court if someone gets around it
- thw_9a83c 1y ago
  ...could be the whole point. They want to make the user to be the indemnifying party to themself. Basically getting rid of any responsibility.
  - vacuity 1y ago
    To be fair, what do you suppose they should do instead? ID isn't popular at all. I don't think the profit motive here is the same as Instagram's, with the latter putting up a front about child safety but making big money from children.
    - thw_9a83c 1y ago
      I agree that having addicted kids on Character.AI is probably less lucrative than having them on Instagram or TikTok. In any case, there is a general problem on the internet regarding how to allow people to remain reasonably anonymous while keeping children out of certain services.
    - BoorishBears 1y ago
      They're gearing up to do ID too (added Persona mentions to their TOS) The additional context here is that Google acquired them to get back Noam Shazeer and took some other members of their technical staff. So the current company is pretty much shambling along after having served its purpose to all stakeholders, and Google probably doesn't really care more than avoiding any sort of liability.
    - subscribed 1y ago
      Current iteration of of ID-based checks is both awful (Discord ID leak proves how dangerous) and hostile (there's high chance this data is being sold already as we speak). It's also laughably simple to circumvent in many cases. Good, privacy preserving age check could be implemented instead, but they won't because it's not about keeping the kids away (and never was). And your example with Instagram is great because they've been caught running experiments actively harming children.
alyxya 1y ago
Deterring kids is still valuable. A lot of them won’t bother with figuring out how to get around the automated detection.
- dylan604 1y ago
  Deterring kids just makes the kids that much more determined. What could possibly be so bad that the adults don't want the kids to use it? Let's find out...
  - gilleain 1y ago
    As the Simpson's quote goes : "Oh Ralphy, why are you always so curious about Daddy's Forbidden Cupboard of Mysteries? ..."
  - ambicapter 1y ago
    It makes _some_ kids more determined, most kids probably don't care that much.
  - mcmoor 1y ago
    Sometimes it's what we want! Especially if that determination results in them training inventive mind to hack. Most of us 20 years ago learnt to navigate our way in the network because of lots of restrictions and obfuscations built-in. Although it'd still be lame if it's so easy to break that someone can share the key, and then the kids don't need to learn anything.
- seanhunter 1y ago
  That’s presumably why age-based deterrents have worked so well at preventing kids from going to age-restricted movies, drinking underage …
LiquidSky 1y ago
Nah, you just have to have some prompt that unknowingly reveals that the user has ever engaged with Roblox in any way, which leads to an instant block.
- zahlman 1y ago
  ... But those people will turn 18 eventually... ?
  - dylan604 1y ago
    that's the price you pay for playing Roblox
- fgonzag 1y ago
  Someone who started using Roblox on its release date in 2006 at let's say 6 years old is now 25 years old.
deleted 1y ago
[deleted]
zzo38computer 1y ago
I would expect that "based on conversations and interactions on the platform" would result in both false positives and false negatives, and that "information from any connected social media accounts" might cause additional problems (possibly including that some people being unable to get in at all, or being able to get in once and then unable to access for reasons that are not explained). (It won't affect me because I do not use it, but it might affect someone who does try to use it)
makeitdouble 1y ago
This is not in the spirit of this law, and will probably irk some, but... If a kid thinks like an adult, behaves like an adult and can't be distinguished from an adult from their online presence, let them use the chatbot. On the other end, I wish they'd flag immature adults as kids as well. 18 is an arbitrary number, and if we have more appropriate ways to judge if someone is ready or not (assuming their check is worth its salt), it should be fine to defer to that. It's not like they're going to a bar to do tequila shots or scam retirees for insurance money.

add-sub-mul-div 1y ago

A model would also have to account for the emotionally stunted adults using an AI bf/gf service.

gs17 1y ago
I doubt they care that much, since the cost of a false positive is that user having to verify their age and the alternative is all users having to verify their age.
BoorishBears 1y ago
The number 1 reason they're doing this is so they can loosen the clamps on NSFW/romance, and they've previously released models targeted at adults intended to improve performance at that.
- gs17 1y ago
  Especially now that OpenAI is entering that space officially by the end of the year.

gs17 1y ago

I don't think the "use technology to detect underage users" approach has ever worked well for its stated intent (it works okay for being able to say you're "doing something" about the problem while not losing as many users), but it's slightly better than mandatory ID for everyone.

mothballed 1y ago
It's just a way to mitigate being sued when inevitably the lawsuits pour in that the AI-assisted harmful decisions that were made totally weren't shared fault of the parents, other external environment, and/or unlucky genetics.
- wat10000 1y ago
  Right, the stated intent may be to block underage users, but the actual intent is to Do Something so that you can't be accused of Not Doing Something.
- SoftTalker 1y ago
  And certainly not the fault of the always conscientious tech companies who are actually the ones running the services.
moi2388 1y ago
Mandatory ID wouldn’t be that bad if implemented claims-based, from within the browser. So that the browser could ask: Hey government, is this user above 18? And the government just responds with yes or no. Sites would then only need to whitelist browsers with this check an merely see an “allowed” claim for whatever type of site they have. Parents or users could use parental controls to “allow” or “block” certain categories, meaning website creators wouldn’t even get any info on the user.

BoorishBears 1y ago

They don't care: if they put in robust sounding guards and openly state "chatbots are not for kids", they can point to them in court if someone gets around it

thw_9a83c 1y ago
...could be the whole point. They want to make the user to be the indemnifying party to themself. Basically getting rid of any responsibility.
- vacuity 1y ago
  To be fair, what do you suppose they should do instead? ID isn't popular at all. I don't think the profit motive here is the same as Instagram's, with the latter putting up a front about child safety but making big money from children.
  - thw_9a83c 1y ago
    I agree that having addicted kids on Character.AI is probably less lucrative than having them on Instagram or TikTok. In any case, there is a general problem on the internet regarding how to allow people to remain reasonably anonymous while keeping children out of certain services.
  - BoorishBears 1y ago
    They're gearing up to do ID too (added Persona mentions to their TOS) The additional context here is that Google acquired them to get back Noam Shazeer and took some other members of their technical staff. So the current company is pretty much shambling along after having served its purpose to all stakeholders, and Google probably doesn't really care more than avoiding any sort of liability.
  - subscribed 1y ago
    Current iteration of of ID-based checks is both awful (Discord ID leak proves how dangerous) and hostile (there's high chance this data is being sold already as we speak). It's also laughably simple to circumvent in many cases. Good, privacy preserving age check could be implemented instead, but they won't because it's not about keeping the kids away (and never was). And your example with Instagram is great because they've been caught running experiments actively harming children.

alyxya 1y ago

Deterring kids is still valuable. A lot of them won’t bother with figuring out how to get around the automated detection.

dylan604 1y ago
Deterring kids just makes the kids that much more determined. What could possibly be so bad that the adults don't want the kids to use it? Let's find out...
- gilleain 1y ago
  As the Simpson's quote goes : "Oh Ralphy, why are you always so curious about Daddy's Forbidden Cupboard of Mysteries? ..."
- ambicapter 1y ago
  It makes _some_ kids more determined, most kids probably don't care that much.
- mcmoor 1y ago
  Sometimes it's what we want! Especially if that determination results in them training inventive mind to hack. Most of us 20 years ago learnt to navigate our way in the network because of lots of restrictions and obfuscations built-in. Although it'd still be lame if it's so easy to break that someone can share the key, and then the kids don't need to learn anything.
seanhunter 1y ago
That’s presumably why age-based deterrents have worked so well at preventing kids from going to age-restricted movies, drinking underage …

LiquidSky 1y ago

Nah, you just have to have some prompt that unknowingly reveals that the user has ever engaged with Roblox in any way, which leads to an instant block.

zahlman 1y ago
... But those people will turn 18 eventually... ?
- dylan604 1y ago
  that's the price you pay for playing Roblox
fgonzag 1y ago
Someone who started using Roblox on its release date in 2006 at let's say 6 years old is now 25 years old.

deleted 1y ago

[deleted]

zzo38computer 1y ago

I would expect that "based on conversations and interactions on the platform" would result in both false positives and false negatives, and that "information from any connected social media accounts" might cause additional problems (possibly including that some people being unable to get in at all, or being able to get in once and then unable to access for reasons that are not explained). (It won't affect me because I do not use it, but it might affect someone who does try to use it)

makeitdouble 1y ago

This is not in the spirit of this law, and will probably irk some, but... If a kid thinks like an adult, behaves like an adult and can't be distinguished from an adult from their online presence, let them use the chatbot. On the other end, I wish they'd flag immature adults as kids as well. 18 is an arbitrary number, and if we have more appropriate ways to judge if someone is ready or not (assuming their check is worth its salt), it should be fine to defer to that. It's not like they're going to a bar to do tequila shots or scam retirees for insurance money.