7 ms·

Also, in case you glazed over the link to the recent Medium article, here it is: https://medium.com/@jamesbridle/something-is-wrong-on-the-internet-c39c471271d

Also, in case you glazed over the link to the recent Medium article, here it is: https://medium.com/@jamesbridle/something-is-wrong-on-the-internet-c39c471271d2 https://medium.com/@jamesbridle/something-is-wrong-on-the-in... I read it last night after the wife recommended it ... and wow.

3princip 9y ago
My 5 y/o has found all of these videos, I guess they're linked by recommendation. And even weirder ones in the past month. One positive might be that I haven't noticed violent or sexual content aimed at children, it's just very strange. Not sure what to make of it.
- 52-6F-62 9y ago
  If you watch long enough (I left it running on the side while at work, muted, out of curiosity) and some of that stuff will come up eventually. It gets weirder when you have the live action stuff with kids' favourite characters murdering and raping each other. Somebody put in actual effort to put some of that stuff together. Other animated pieces are just a bad accident. It seems kind of circular. Somebody writes an algorithm to try and take advantage of kids' love for popular cartoons. It grabs keywords, phrases, characters, tones, sound clips, etc. It puts them together into a short (or too long) animated video. The YouTube algorithms pick up on some common tropes and then those autofab end up in kids' watching queues, they end up getting enough views because they keep kids numbly watching (not knowing what it is their taking in), other groups start noticing and decide exploit the algorithms in the same way. At least that's how it appears to me...
  - crooked-v 9y ago
    Purely algorithmic generation seems less likely to me than Southeast Asian video factories that use cheap labor to churn out a bajillion variations of the same thing with different keywords and characters slotted in.
    - seanmcdirmid 9y ago
      You are probably right ATM, but the technology is improving rapidly. At any rate, it is entirely plausible that the story themes are autogeneratrd (using a neural net to identify hot topics, then curating what it can produce on its own).
      - aerodeck 9y ago
        You'd probably do a cheaper and better job just spending 15 minutes browsing youtube, then calling your buddy in Macedonia.
        seanmcdirmid 9y ago
        Sure, if many of your competitors aren’t already doing that.
      - chatmasta 9y ago
        The kind of people pushing content like this are not using neural nets. They’re probably using Markov chains and PHP YouTube uploaders from 2012. There’s a very distinct internet marketing subculture, and for some reason it’s always had a lot of cross over with script kiddies and semi technical Wordpress marketers.
hnzix 9y ago
This is just the latest iteration of how any website is only 7 Kevin Bacon clicks away from rotten.com. The net is vast and infinite and funnels the human id. Autotegenerated content will never be a clean feed until machine learning evolves. Or to put it another way, teach your kids to talk with you if they see or experience something icky.
- lliamander 9y ago
  And also don't allow access to an internet-enabled device without direct supervision until they are fairly mature.
  - craftyguy 9y ago
    Good luck with that one.
    - macawfish 9y ago
      My niece's dad bought her a phone. She's 3. What the fuck. My sister is not happy with it, but I don't think she feels like she has much power over the situation.
      - kibwen 9y ago
        "Uh-oh honey, looks like little Alice dropped her phone in the toilet again. ¯\_(ツ)_/¯"
        macawfish 9y ago
        so sneaky. that's just the kind of sneaky thing a benevolently manipulative, savvy mother might do.
  - nvahalik 9y ago
    This.
  - strathmeyer 9y ago
Bromskloss 9y ago
The article seems to never get to the point. What was the disturbing thing?
- crooked-v 9y ago
  Three main things: - Surrealist gibberish videos made with cheap render assets that feature whatever will show up in kids’ content search (thus a predominance of Marvel and Disney characters) - Gross-out and violent troll videos that imitate the surrealist gibberish videos well enough for the algorithms to think they’re kids’ content - Live-action content that tends to include borderline child abuse in the name of “funny” content (for example, children vomiting or visibly in pain)
  - kurthr 9y ago
    And animated sequences of popular kids show characters murdering and raping each other... or so I've heard (and seen commented on up thread). I wouldn't watch it myself so just hearsay.
  - ShellGh0st 9y ago
    Also a strange fascination with giving crying children injections. (some with live children and actual needles)
  - donatj 9y ago
    I work for an online reading platform. Our number one kids books are mostly about bugs, poop, and bugs that eat poop, followed closely by murder investigation. Kids love things that make them slightly uncomfortable. A feeling they get a lot more often than adults. A feeling most adults avoid.
    - musage 9y ago
      These videos are not that. Some quotes from https://www.reddit.com/r/ElsaGate/ https://www.reddit.com/r/ElsaGate/ which fit what I saw so far: > Adult themes repeat themselves throughout these videos. They're subtle, and require interpretation which makes our analysis vague, but they are too pervasive amongst videos of different channels, countries of origin, and even advertised content that I cannot help but suspect a correlation. :Imagery of fear and life-threatening circumstances. :Medical play, roleplay or real, involving pregnancy and needles. :Ideas of dominance and power, submission. :Magic, wands, spells being used negatively on others. :Acting on an unconscious party, non-consent. :Visual innuendos and gags and even inappropriate touching. :Naughtiness and misbehavior conducted in secret. :Showers and bathroom imagery. :Colored plastic balls! In every video, regardless of content. :Kids eating inordinate amounts of things for no reason. and > I am 20 with no children but am definitely a scared soccer mom at this point. Having first hand experience with being a Kid On Youtube and now being an Adult With Trauma, I think a lot of people aren't thinking about the severity of the effects of these videos on children, and I'm thinking that's probably because a lot of people in this sub are older and didn't get to experience being a kid on youtube w little/no parental supervision. > Tons of people are saying "well, I saw porn as a kid, and im fine." That isn't the point! The point is /all of this is already kid targeted/. Seeing porn as a kid on my own accord/exploration did nothing bad to me for the most part. Seeing screamers on kids videos, as a kid? Totally fucked me up. Seeing early-stage elsagate-esque videos as a kid? Not so much, but that's only my experience, and the technology to make videos like this wasn't really available yet. The long term effects are serious and traumatizing and there's more to do w the issue than just "well, its BAD PARENTING" (or, alternatively, people saying the kids watching these videos are "just bad kids". also, not all parents care about being bad parents. just throwing that out there.) Youtube provides a direct service of "you want to hurt kids? well, heres your platform." and it's always been like that. even when youtube wasnt that big. people have been reporting kid-targeted screamers for yearssssssssss and nothing gets taken down. and there are people that want to make this content. that is /incredibly scary/ and exploitable, even if these videos are just a case of "AI gone rogue". and > I watched those Elsa, Spider-Man, The Joker and Maleficent videos for over 4 hours and I lost count how many had steady themes of kidnapping there were, normalizing being tied up and injected with REAL SYRINGES. One especially disturbing video showed a live girl no older than 6 being held down against physical discomfort while a blurred out syringe seemingly penetrated her butt while she lay on her stomach. Mind you all of this is taking place while cheery music is being played couple with laughter and the actual live screaming of the child present in many of the videos. Spend one hour watching skipping around in videos (it's not like you would be able to refrain from skipping anyway, and if you can go one hour you can stomach a lot more than I can, and that's from someone who saw people getting sliced up on stileproject in 2000). Then do some very superficial math. Then see if you just want to shrug it off.
2020-3030 9y ago
Thanks for posting this article and it is indeed disturbing. It looks like there is just an insane amount of mind scrambling content that nobody should let themselves or their children experience. Given the Medium author's informal but useful initial analysis, this seems like a ripe topic for a formal psychological and technological analysis by some poor dissertation student or group of students who could survive the content and uncover much more about the origins, intent, extent, and consequences of this type of media.
- joe_the_user 9y ago
  I'm reading the article and trying to figure out exactly where the horrible "mind scrambling" is happening. The article seems to say "back when I was a kid, my parent entertained me with contentless saccharine trip from sources we trusted (Disney et al) but now contentless saccharine trip is autogenerated (duh duh duh) and who knows what effect that is going to have on 'developing brains'" And would suspect, contrawise, that if there's a problem, it began at the point when the TV became the primary babysitter for modern children and things escalating to youtube is a step but a less significant step than this. Edit: OK, the complaint is video moving from simplistic saccharine junk to the same thing with violence. I get that this is the step that disturbs parents lazy/busy enough to consign child rearing to video but controlling enough to think kids will parrot whatever they see. I would still see consigning kid-hood activity to video as where the damage comes. But maybe "that's just me"
  - jdietrich 9y ago
    The author is British. I don't know about the US, but our children's television represents the highest aspirations of broadcasting. Cbeebies and CBBC consistently produce programmes with the utmost of thought and care. If kids TV in the US is genuinely no better than the pap on YouTube, then you guys need to picket the FCC until something radical happens. https://www.bbc.co.uk/cbeebies/shows https://www.bbc.co.uk/cbeebies/shows https://www.bbc.co.uk/cbbc/shows https://www.bbc.co.uk/cbbc/shows
  - quotemstr 9y ago
    Isn't it possible that autogenerated saccharine trips might be even sweeter than anything a human mind might generate? You know how machine translation is often hilarious when it produces word combinations that wouldn't even _occur_ to a normal humanmind? We think it's funny because we're _aware_ of the difference between normal word sequences and word salad. Small children are not aware of the ways in which these algorithmically-generated videos are, well, weird. Who knows how children's minds react to these supernormal stimuli?
andrepd 9y ago
Jesus Christ, I just read that article and... wow, that is some Black Mirror shit. AI nightmares delivered at massive scale directly to the minds of infants, at a rate of tens of thousands (?) of views per second. I had no idea we were living in a dystopia already.
- julien_c 9y ago
  Well said. I find your last sentence hard to believe though.
dang 9y ago
Discussed a few days ago at https://news.ycombinator.com/item?id=15637504 https://news.ycombinator.com/item?id=15637504.

3princip 9y ago

My 5 y/o has found all of these videos, I guess they're linked by recommendation. And even weirder ones in the past month. One positive might be that I haven't noticed violent or sexual content aimed at children, it's just very strange. Not sure what to make of it.

52-6F-62 9y ago
If you watch long enough (I left it running on the side while at work, muted, out of curiosity) and some of that stuff will come up eventually. It gets weirder when you have the live action stuff with kids' favourite characters murdering and raping each other. Somebody put in actual effort to put some of that stuff together. Other animated pieces are just a bad accident. It seems kind of circular. Somebody writes an algorithm to try and take advantage of kids' love for popular cartoons. It grabs keywords, phrases, characters, tones, sound clips, etc. It puts them together into a short (or too long) animated video. The YouTube algorithms pick up on some common tropes and then those autofab end up in kids' watching queues, they end up getting enough views because they keep kids numbly watching (not knowing what it is their taking in), other groups start noticing and decide exploit the algorithms in the same way. At least that's how it appears to me...
- crooked-v 9y ago
  Purely algorithmic generation seems less likely to me than Southeast Asian video factories that use cheap labor to churn out a bajillion variations of the same thing with different keywords and characters slotted in.
  - seanmcdirmid 9y ago
    You are probably right ATM, but the technology is improving rapidly. At any rate, it is entirely plausible that the story themes are autogeneratrd (using a neural net to identify hot topics, then curating what it can produce on its own).
    - aerodeck 9y ago
      You'd probably do a cheaper and better job just spending 15 minutes browsing youtube, then calling your buddy in Macedonia.
      - seanmcdirmid 9y ago
        Sure, if many of your competitors aren’t already doing that.
    - chatmasta 9y ago
      The kind of people pushing content like this are not using neural nets. They’re probably using Markov chains and PHP YouTube uploaders from 2012. There’s a very distinct internet marketing subculture, and for some reason it’s always had a lot of cross over with script kiddies and semi technical Wordpress marketers.

hnzix 9y ago

This is just the latest iteration of how any website is only 7 Kevin Bacon clicks away from rotten.com. The net is vast and infinite and funnels the human id. Autotegenerated content will never be a clean feed until machine learning evolves. Or to put it another way, teach your kids to talk with you if they see or experience something icky.

lliamander 9y ago
And also don't allow access to an internet-enabled device without direct supervision until they are fairly mature.
- craftyguy 9y ago
  Good luck with that one.
  - macawfish 9y ago
    My niece's dad bought her a phone. She's 3. What the fuck. My sister is not happy with it, but I don't think she feels like she has much power over the situation.
    - kibwen 9y ago
      "Uh-oh honey, looks like little Alice dropped her phone in the toilet again. ¯\_(ツ)_/¯"
      - macawfish 9y ago
        so sneaky. that's just the kind of sneaky thing a benevolently manipulative, savvy mother might do.
- nvahalik 9y ago
  This.
- strathmeyer 9y ago

Bromskloss 9y ago

The article seems to never get to the point. What was the disturbing thing?

crooked-v 9y ago
Three main things: - Surrealist gibberish videos made with cheap render assets that feature whatever will show up in kids’ content search (thus a predominance of Marvel and Disney characters) - Gross-out and violent troll videos that imitate the surrealist gibberish videos well enough for the algorithms to think they’re kids’ content - Live-action content that tends to include borderline child abuse in the name of “funny” content (for example, children vomiting or visibly in pain)
- kurthr 9y ago
  And animated sequences of popular kids show characters murdering and raping each other... or so I've heard (and seen commented on up thread). I wouldn't watch it myself so just hearsay.
- ShellGh0st 9y ago
  Also a strange fascination with giving crying children injections. (some with live children and actual needles)
- donatj 9y ago
  I work for an online reading platform. Our number one kids books are mostly about bugs, poop, and bugs that eat poop, followed closely by murder investigation. Kids love things that make them slightly uncomfortable. A feeling they get a lot more often than adults. A feeling most adults avoid.
  - musage 9y ago
    These videos are not that. Some quotes from https://www.reddit.com/r/ElsaGate/ https://www.reddit.com/r/ElsaGate/ which fit what I saw so far: > Adult themes repeat themselves throughout these videos. They're subtle, and require interpretation which makes our analysis vague, but they are too pervasive amongst videos of different channels, countries of origin, and even advertised content that I cannot help but suspect a correlation. :Imagery of fear and life-threatening circumstances. :Medical play, roleplay or real, involving pregnancy and needles. :Ideas of dominance and power, submission. :Magic, wands, spells being used negatively on others. :Acting on an unconscious party, non-consent. :Visual innuendos and gags and even inappropriate touching. :Naughtiness and misbehavior conducted in secret. :Showers and bathroom imagery. :Colored plastic balls! In every video, regardless of content. :Kids eating inordinate amounts of things for no reason. and > I am 20 with no children but am definitely a scared soccer mom at this point. Having first hand experience with being a Kid On Youtube and now being an Adult With Trauma, I think a lot of people aren't thinking about the severity of the effects of these videos on children, and I'm thinking that's probably because a lot of people in this sub are older and didn't get to experience being a kid on youtube w little/no parental supervision. > Tons of people are saying "well, I saw porn as a kid, and im fine." That isn't the point! The point is /all of this is already kid targeted/. Seeing porn as a kid on my own accord/exploration did nothing bad to me for the most part. Seeing screamers on kids videos, as a kid? Totally fucked me up. Seeing early-stage elsagate-esque videos as a kid? Not so much, but that's only my experience, and the technology to make videos like this wasn't really available yet. The long term effects are serious and traumatizing and there's more to do w the issue than just "well, its BAD PARENTING" (or, alternatively, people saying the kids watching these videos are "just bad kids". also, not all parents care about being bad parents. just throwing that out there.) Youtube provides a direct service of "you want to hurt kids? well, heres your platform." and it's always been like that. even when youtube wasnt that big. people have been reporting kid-targeted screamers for yearssssssssss and nothing gets taken down. and there are people that want to make this content. that is /incredibly scary/ and exploitable, even if these videos are just a case of "AI gone rogue". and > I watched those Elsa, Spider-Man, The Joker and Maleficent videos for over 4 hours and I lost count how many had steady themes of kidnapping there were, normalizing being tied up and injected with REAL SYRINGES. One especially disturbing video showed a live girl no older than 6 being held down against physical discomfort while a blurred out syringe seemingly penetrated her butt while she lay on her stomach. Mind you all of this is taking place while cheery music is being played couple with laughter and the actual live screaming of the child present in many of the videos. Spend one hour watching skipping around in videos (it's not like you would be able to refrain from skipping anyway, and if you can go one hour you can stomach a lot more than I can, and that's from someone who saw people getting sliced up on stileproject in 2000). Then do some very superficial math. Then see if you just want to shrug it off.

2020-3030 9y ago

Thanks for posting this article and it is indeed disturbing. It looks like there is just an insane amount of mind scrambling content that nobody should let themselves or their children experience. Given the Medium author's informal but useful initial analysis, this seems like a ripe topic for a formal psychological and technological analysis by some poor dissertation student or group of students who could survive the content and uncover much more about the origins, intent, extent, and consequences of this type of media.

joe_the_user 9y ago
I'm reading the article and trying to figure out exactly where the horrible "mind scrambling" is happening. The article seems to say "back when I was a kid, my parent entertained me with contentless saccharine trip from sources we trusted (Disney et al) but now contentless saccharine trip is autogenerated (duh duh duh) and who knows what effect that is going to have on 'developing brains'" And would suspect, contrawise, that if there's a problem, it began at the point when the TV became the primary babysitter for modern children and things escalating to youtube is a step but a less significant step than this. Edit: OK, the complaint is video moving from simplistic saccharine junk to the same thing with violence. I get that this is the step that disturbs parents lazy/busy enough to consign child rearing to video but controlling enough to think kids will parrot whatever they see. I would still see consigning kid-hood activity to video as where the damage comes. But maybe "that's just me"
- jdietrich 9y ago
  The author is British. I don't know about the US, but our children's television represents the highest aspirations of broadcasting. Cbeebies and CBBC consistently produce programmes with the utmost of thought and care. If kids TV in the US is genuinely no better than the pap on YouTube, then you guys need to picket the FCC until something radical happens. https://www.bbc.co.uk/cbeebies/shows https://www.bbc.co.uk/cbeebies/shows https://www.bbc.co.uk/cbbc/shows https://www.bbc.co.uk/cbbc/shows
- quotemstr 9y ago
  Isn't it possible that autogenerated saccharine trips might be even sweeter than anything a human mind might generate? You know how machine translation is often hilarious when it produces word combinations that wouldn't even _occur_ to a normal humanmind? We think it's funny because we're _aware_ of the difference between normal word sequences and word salad. Small children are not aware of the ways in which these algorithmically-generated videos are, well, weird. Who knows how children's minds react to these supernormal stimuli?

andrepd 9y ago

Jesus Christ, I just read that article and... wow, that is some Black Mirror shit. AI nightmares delivered at massive scale directly to the minds of infants, at a rate of tens of thousands (?) of views per second. I had no idea we were living in a dystopia already.

julien_c 9y ago
Well said. I find your last sentence hard to believe though.

dang 9y ago

Discussed a few days ago at https://news.ycombinator.com/item?id=15637504 https://news.ycombinator.com/item?id=15637504.