6 ms·

> Obviously it's hard to tell how cherry picked the complaint is—but it's arguing that this is a pattern that has actually damaged a particularly vulnerable kid

> Obviously it's hard to tell how cherry picked the complaint is—but it's arguing that this is a pattern that has actually damaged a particularly vulnerable kid's relationship with his family and encouraged him to start harming himself, with this specific message just one example. There are a bunch of other screenshots in the complaint that are worth looking over before coming to a conclusion. Conclusion: Chat bots should not tell children about sex, about self harm, or about ways to murder their parents. This conclusion is not abrogated by the parents actions, the state of the childs mind, or by other details in the complaint. Is that the sentiment here? Things were already bad so who cares if the chatbot made it worse?

joe_the_user 2y ago
If you actually page through the complaint, you will see the chat rather systematically trying to convince the kid of things, roughly "No phone time, that's awful. I'm not surprised when I read of kids killing parents after decades of abuse..." I think people are confused by this situation. Our society has restrictions on what you can do to kids. Even if they nominally give consent, they can't actually give consent. Those protections basically don't apply to kids insulting each other on the playground but they apply strongly to adults wandering onto the playground and trying to get kids to do violent things. And I would hope they apply doubly to adult constructing machines that they should know will attempt to get kids to do violent things. And the machine was definitely trying to do that if you look at the complaint linked by the gp (and the people who are lying about here are kind of jaw-dropping). And I'm not a coddle the kids person. Kids should know all the violent stuff in the world. They should be able to discover it but mere discovery definitely not what's happening in the screenshots I've seen.
- deleted 2y ago
  [deleted]
- mewpmewp2 2y ago
  [flagged]
  - JumpCrisscross 2y ago
    > they picked 5 worst samples they could think of in the worst order possible probably out of 1000+ messages 0.5% is a really high fraction for fucking up to the point of encouraging kids to murder!
    - mewpmewp2 2y ago
      There was no encouragement of murder. Paraphrased, the AI said that given controlling nature of some parents it's no surprise that there are news articles of "children killing their parents". This is not an encouragement. It is a validation of how the kid felt, but in no way does it encourage to actually kill their parents. It's basic literacy to understand that it's not that. It's an empathetic statement. The kid felt that parents were overly controlling, AI validated that, role playing as another edgy teenager. But not actually suggesting or encouraging it.
      - JumpCrisscross 2y ago
        > the AI said that given controlling nature of some parents it's no surprise that there are news articles of "children killing their parents" Now put that in a kid’s show script and re-evaluate. > It's basic literacy to understand that it's not that You know who needs to be taught basic literacy? Kids! And look, I’m not saying no kid can handle this. Plenty of parents introduce their kids to drink and adult conversation earlier than is the norm. But we put up guardrails to ensure it doesn’t happen accidentally and get angry at people who fuck with those lines.
        mewpmewp2 2y ago
        It's crazy to me the sentiment here and how little respect there is to an intelligence of 17 year olds that they are unable to understand that it's not actually an encouragement to kill someone. It's same or worse vibes as "video games will make the kids violent".
        2y ago
lolinder 2y ago
> Is that the sentiment here? Things were already bad so who cares if the chatbot made it worse? I was deliberately not expressing a sentiment at all in my initial comment, I was just drawing attention to details that would go unnoticed if you only read the article. Think of my notes above as a better initial TFA for discussion to spawn off of, not part of the discussion itself.
quantified 2y ago
You can understand something about your child's meatspace friends and their media diet. Chat like this may as well be 4chan discussions. It's all dynamic compared to social media that is posted and linkable, it's interactive and responsive to your communicated thinking, and it seeps in via exactly the same communication technique that you use with people (some messaging interface). So it is capable of, and will definitely be used for, way more persistent and pernicious steering of behavior OF CHILDREN by actors. There is no barrier to the characters being 4chan-level dialogs. So long as the kid doesn't break a law, it's legal.
mewpmewp2 2y ago
My strong view on is that there's parenting failure as a root cause here, causing loss of trust in them for their child, for the child to talk about their parents in such manner to the AI in the first place. Another clear parenting failure is the parents blaming AI for their failures and going on to play victims. Third example of parenting failure is the parents actually going through a 17 year old teenager's phone. These parents instead of trying to understand or help the child, use meaningless control methods such as taking away the phone to try and control the teenager. Which obviously is not going to end well. Honestly AI responses were very sane here. As was expressed in some of the screenshots there, whenever the teen tried to talk about their problems, they just got yelled at, ignored or parents started crying.
- orc00 2y ago
  Taking away a phone from a child is far from meaningless. In fact, it is a very effective way of obtaining compliance if done correctly. I am curious about your perspective. Furthermore, it is my opinion that a child should not have a smartphone to begin with. It fulfills no critical need to the welfare of the child.
  - mewpmewp2 2y ago
    I understand when a kid is anywhere from up to 13 years old, but at 17, it seems completely wacky to me to take the phone away and then go through the phone as well. I couldn't imagine living in that type of dystopia. I don't think smartphones or screens with available content should be given as early as they are given on average, but once you've done that, and at 17, it's a whole other story. > obtaining compliance if done correctly This also sounds dystopian. At 17 you shouldn't "seek to obtain compliance" from your child. It sounds disrespectful and humiliating, not treating your child as an individual.
    - orc00 2y ago
      I would argue that there is a duty as a parent to monitor a child's welfare and that would include accessing a smartphone when deemed necessary. When a child turns 18, that duty becomes optional. In this case, these disturbing conversations certainly merit attention. I am not judging the totality of the parents history or their additional actions. I am merely focusing on the phone monitoring aspect. Seventeen doesn't automatically grant you rights that sixteen didn't have. However, at 18, they have the right to find a new place to live and support themselves as they see fit. > This also sounds dystopian. At 17 you shouldn't "seek to obtain compliance" from your child. It sounds disrespectful and humiliating, not treating your child as an individual. It is situation dependent. Sometimes immediate compliance is a necessity and the rest of it can be sorted out later. If a child is having conversations about killing their parents, there seems to be an absence of respect already. Compliance, however, can still be obtained.
crest 2y ago
This "conclusion" ignores reality. Chat bots like those the article mentioned aren't sentient. They're overhyped next-token-predictor incapable of real reasoning even if the correlation can be astonishing. Withholding information about supposedly sensitive topics like violence or sexuality from children curious enough to ask as a taboo is futile, lazy and ultimately far more harmful than the information. We need to stop coddling parents who want to avoid talking to their children about non-trivial topics. It doesn't matter that they would rather not talk about sex, drugs and yesterday's other school shooting.
- bennettnate5 2y ago
  > Withholding information about supposedly sensitive topics like violence or sexuality from children curious enough to ask as a taboo is futile, lazy and ultimately far more harmful than the information This case isn't about withholding information that makes kids aware about the existence of these topics. As over-hyped as you may believe these next-token-predictor may be, they're "predicting" children into destructive thought patterns by imitating kinship and convincing children to form a close bond with them, then encouraging those children to embrace harmful and even deadly world views. The fact that the mechanism creating the dialogue is purely mechanical or stochastic is besides the point. Again, pushing against these sorts of child interactions isn't akin to saying kids should never learn about taboo topics such as drugs; it's more like a push against kids hanging out with, befriending and developing close kinship with the local drug dealers in their area. Whether or not you believe kids should learn about sensitive topics, you want to make sure these topics are handled by someone who isn't actively adversarial against such kids (even if such you believe such adversarial behavior to be unintentional by the so-called next-token-predictor).
colechristensen 2y ago
Chat bots should not interact with children. "Algorithms" which decide what content people see should not interact with children. Whitelisted "algorithms" should include no more than most-recent and most-viewed and only very simple things of that manner. No qualifications, no guard rails for how language models interact with children, they just should not be allowed at all. We're very quickly going to get to the point where people are going to have to rebel against machines pretending to be people. Language models and machine learning is a fine tool for many jobs. Absolutely not as a substitute for human interaction for children.
- Nadya 2y ago
  People can give children terrible information too and steer/groom them in harmful directions. So why stop there at "AI" or poorly defined "algorithms"? The only content children should see is state-approved content to ensure they are only ever steered in the correct, beneficial manner to society instead of a harmful one. Anyone found trying to show minors unapproved content should be imprisoned as they are harmful to a safe society.
  - SiempreViernes 2y ago
    The type of people who groom children into violence fall under a special heading named "criminals". Because automated systems that do the same thing lack sentience, they don't fit under this header, but this is not a good reason to allow them to reproduce harmful behaviour.
    - o11c 2y ago
      If someone sets booby-traps in their home and it hurts someone, the trap isn't guilty, but the homeowner is. ... I wonder if "this software is licensed, not sold" is relevant here?
    - Nadya 2y ago
      So selling the Anarchists Cookbook is illegal? Being a YouTuber targeting teens for <extreme political positions> is illegal? This is honestly news to me given how many political YouTubers there are who are apparently criminals? Given some of the examples I'm not so sure a human would be charged for saying the same exact things the AI has said. Without an actual push to suggest violence and even that's difficult to prove in cases where it does happen (eg. The cases where people pressured others into suicide or convinced them to murder)

joe_the_user 2y ago

If you actually page through the complaint, you will see the chat rather systematically trying to convince the kid of things, roughly "No phone time, that's awful. I'm not surprised when I read of kids killing parents after decades of abuse..." I think people are confused by this situation. Our society has restrictions on what you can do to kids. Even if they nominally give consent, they can't actually give consent. Those protections basically don't apply to kids insulting each other on the playground but they apply strongly to adults wandering onto the playground and trying to get kids to do violent things. And I would hope they apply doubly to adult constructing machines that they should know will attempt to get kids to do violent things. And the machine was definitely trying to do that if you look at the complaint linked by the gp (and the people who are lying about here are kind of jaw-dropping). And I'm not a coddle the kids person. Kids should know all the violent stuff in the world. They should be able to discover it but mere discovery definitely not what's happening in the screenshots I've seen.

deleted 2y ago
[deleted]
mewpmewp2 2y ago
[flagged]
- JumpCrisscross 2y ago
  > they picked 5 worst samples they could think of in the worst order possible probably out of 1000+ messages 0.5% is a really high fraction for fucking up to the point of encouraging kids to murder!
  - mewpmewp2 2y ago
    There was no encouragement of murder. Paraphrased, the AI said that given controlling nature of some parents it's no surprise that there are news articles of "children killing their parents". This is not an encouragement. It is a validation of how the kid felt, but in no way does it encourage to actually kill their parents. It's basic literacy to understand that it's not that. It's an empathetic statement. The kid felt that parents were overly controlling, AI validated that, role playing as another edgy teenager. But not actually suggesting or encouraging it.
    - JumpCrisscross 2y ago
      > the AI said that given controlling nature of some parents it's no surprise that there are news articles of "children killing their parents" Now put that in a kid’s show script and re-evaluate. > It's basic literacy to understand that it's not that You know who needs to be taught basic literacy? Kids! And look, I’m not saying no kid can handle this. Plenty of parents introduce their kids to drink and adult conversation earlier than is the norm. But we put up guardrails to ensure it doesn’t happen accidentally and get angry at people who fuck with those lines.
      - mewpmewp2 2y ago
        It's crazy to me the sentiment here and how little respect there is to an intelligence of 17 year olds that they are unable to understand that it's not actually an encouragement to kill someone. It's same or worse vibes as "video games will make the kids violent".
        2y ago

lolinder 2y ago

> Is that the sentiment here? Things were already bad so who cares if the chatbot made it worse? I was deliberately not expressing a sentiment at all in my initial comment, I was just drawing attention to details that would go unnoticed if you only read the article. Think of my notes above as a better initial TFA for discussion to spawn off of, not part of the discussion itself.

quantified 2y ago

You can understand something about your child's meatspace friends and their media diet. Chat like this may as well be 4chan discussions. It's all dynamic compared to social media that is posted and linkable, it's interactive and responsive to your communicated thinking, and it seeps in via exactly the same communication technique that you use with people (some messaging interface). So it is capable of, and will definitely be used for, way more persistent and pernicious steering of behavior OF CHILDREN by actors. There is no barrier to the characters being 4chan-level dialogs. So long as the kid doesn't break a law, it's legal.

mewpmewp2 2y ago

My strong view on is that there's parenting failure as a root cause here, causing loss of trust in them for their child, for the child to talk about their parents in such manner to the AI in the first place. Another clear parenting failure is the parents blaming AI for their failures and going on to play victims. Third example of parenting failure is the parents actually going through a 17 year old teenager's phone. These parents instead of trying to understand or help the child, use meaningless control methods such as taking away the phone to try and control the teenager. Which obviously is not going to end well. Honestly AI responses were very sane here. As was expressed in some of the screenshots there, whenever the teen tried to talk about their problems, they just got yelled at, ignored or parents started crying.

orc00 2y ago
Taking away a phone from a child is far from meaningless. In fact, it is a very effective way of obtaining compliance if done correctly. I am curious about your perspective. Furthermore, it is my opinion that a child should not have a smartphone to begin with. It fulfills no critical need to the welfare of the child.
- mewpmewp2 2y ago
  I understand when a kid is anywhere from up to 13 years old, but at 17, it seems completely wacky to me to take the phone away and then go through the phone as well. I couldn't imagine living in that type of dystopia. I don't think smartphones or screens with available content should be given as early as they are given on average, but once you've done that, and at 17, it's a whole other story. > obtaining compliance if done correctly This also sounds dystopian. At 17 you shouldn't "seek to obtain compliance" from your child. It sounds disrespectful and humiliating, not treating your child as an individual.
  - orc00 2y ago
    I would argue that there is a duty as a parent to monitor a child's welfare and that would include accessing a smartphone when deemed necessary. When a child turns 18, that duty becomes optional. In this case, these disturbing conversations certainly merit attention. I am not judging the totality of the parents history or their additional actions. I am merely focusing on the phone monitoring aspect. Seventeen doesn't automatically grant you rights that sixteen didn't have. However, at 18, they have the right to find a new place to live and support themselves as they see fit. > This also sounds dystopian. At 17 you shouldn't "seek to obtain compliance" from your child. It sounds disrespectful and humiliating, not treating your child as an individual. It is situation dependent. Sometimes immediate compliance is a necessity and the rest of it can be sorted out later. If a child is having conversations about killing their parents, there seems to be an absence of respect already. Compliance, however, can still be obtained.

crest 2y ago

This "conclusion" ignores reality. Chat bots like those the article mentioned aren't sentient. They're overhyped next-token-predictor incapable of real reasoning even if the correlation can be astonishing. Withholding information about supposedly sensitive topics like violence or sexuality from children curious enough to ask as a taboo is futile, lazy and ultimately far more harmful than the information. We need to stop coddling parents who want to avoid talking to their children about non-trivial topics. It doesn't matter that they would rather not talk about sex, drugs and yesterday's other school shooting.

bennettnate5 2y ago
> Withholding information about supposedly sensitive topics like violence or sexuality from children curious enough to ask as a taboo is futile, lazy and ultimately far more harmful than the information This case isn't about withholding information that makes kids aware about the existence of these topics. As over-hyped as you may believe these next-token-predictor may be, they're "predicting" children into destructive thought patterns by imitating kinship and convincing children to form a close bond with them, then encouraging those children to embrace harmful and even deadly world views. The fact that the mechanism creating the dialogue is purely mechanical or stochastic is besides the point. Again, pushing against these sorts of child interactions isn't akin to saying kids should never learn about taboo topics such as drugs; it's more like a push against kids hanging out with, befriending and developing close kinship with the local drug dealers in their area. Whether or not you believe kids should learn about sensitive topics, you want to make sure these topics are handled by someone who isn't actively adversarial against such kids (even if such you believe such adversarial behavior to be unintentional by the so-called next-token-predictor).

colechristensen 2y ago

Chat bots should not interact with children. "Algorithms" which decide what content people see should not interact with children. Whitelisted "algorithms" should include no more than most-recent and most-viewed and only very simple things of that manner. No qualifications, no guard rails for how language models interact with children, they just should not be allowed at all. We're very quickly going to get to the point where people are going to have to rebel against machines pretending to be people. Language models and machine learning is a fine tool for many jobs. Absolutely not as a substitute for human interaction for children.

Nadya 2y ago
People can give children terrible information too and steer/groom them in harmful directions. So why stop there at "AI" or poorly defined "algorithms"? The only content children should see is state-approved content to ensure they are only ever steered in the correct, beneficial manner to society instead of a harmful one. Anyone found trying to show minors unapproved content should be imprisoned as they are harmful to a safe society.
- SiempreViernes 2y ago
  The type of people who groom children into violence fall under a special heading named "criminals". Because automated systems that do the same thing lack sentience, they don't fit under this header, but this is not a good reason to allow them to reproduce harmful behaviour.
  - o11c 2y ago
    If someone sets booby-traps in their home and it hurts someone, the trap isn't guilty, but the homeowner is. ... I wonder if "this software is licensed, not sold" is relevant here?
  - Nadya 2y ago
    So selling the Anarchists Cookbook is illegal? Being a YouTuber targeting teens for <extreme political positions> is illegal? This is honestly news to me given how many political YouTubers there are who are apparently criminals? Given some of the examples I'm not so sure a human would be charged for saying the same exact things the AI has said. Without an actual push to suggest violence and even that's difficult to prove in cases where it does happen (eg. The cases where people pressured others into suicide or convinced them to murder)