5 ms·

Meta apologises over flood of gore, violence and dead bodies on Instagram

70 points by chrisjj 2y ago

theshrike79 2y ago
> "We have fixed an error that caused some users to see content on their Instagram Reels feed that should not have been recommended" How did that stuff exist on their platform at all?
- pbalau 2y ago
  Users post it?
  - jajko 2y ago
    And nobody really moderates it... pretty sure meta is breaking some laws at least in some jurisdictions but this is nothing new I have in recent year maybe seen a massive increase of various fake posts from unknown groups on facebook. Made up shorts, AI generated fake photos of various weird outrageous claims, which all upon some side check turned out fake but it was not obvious on first glance. I try to report them, mark them as not interesting etc. but they keep popping back without me ever being interested in them. Meta is really shitting on its users and just milking current status, or doesn't care how content on their platforms degrade further, or even quietly allows it. I'd say its the second choice, and it is really by choice. Youtube has similar potential but somehow they manage to sanitize it all well, so clearly its possible.
  - brookst 2y ago
    Right? Not a fan of Meta, but the flip side of this is “how dare those censorious corporate overlords limit what I can post to my account”
  - theshrike79 2y ago
    Shouldn't a multi-billion dollar company have moderation in place to tag content like that? I'm pretty sure if you start posting spoilers about Marvel properties or straight-up pirated content it'll be taken down as fast as possible. But gore? Nah, why bother.
sdwr 2y ago
If this is because of reduced moderation, does that mean the algorithm defaults to shock content? That gets the most attention, and they're artificially suppressing it most of the time? If the attention economy is a race to the bottom, then the most popular content should be stuff that doesn't quite break the public decency rules.
- mc32 2y ago
  Ultimately it’s because users consume shock content; if they didn’t it wouldn’t be surfaced and instead they would get whatever generates views be it cats or rainbows.
  - gosub100 2y ago
    > users survivorship bias. you aren't account for how many users were repulsed or shocked (read: shook up, not a momentary "wtf") and left the platform.
  - cardanome 2y ago
    No, stop blaming the users. Look at this comment: https://news.ycombinator.com/item?id=43204869 https://news.ycombinator.com/item?id=43204869 It clearly shows that passively scrolling on a fresh account will get you gore in no time. Yes, as humans, shock content makes us look but that doesn't mean users want this kind of content. They are being manipulated by these algorithms. Social networks are optimizing for "engagement", meaning ultimately profit, not what users actually want. This is the lazy excuse of every company ever hurting society. Oh, the users want it. No, nobody asked for more gore on Instagram. You are just wanting to save money on moderation.
    - ourmandave 2y ago
      I remember when ISIS posted that beheading video of a journalist, I had to go out of my to avoid seeing it. And that was before Instagram.
    - nullc 2y ago
      I read that comment's "doom scrolling" as implying that it intentionally watched negative-emotion-producing-material. Is this an incorrect reading? If so how does "doom scrolling" differ from "scrolling"?
AlecSchueler 2y ago
Had to delete mine a few weeks ago when I suddenly got a pro-book burning reel, showing Nazis burning books with the caption "They tell you they burned books, bit never which books." It had 30,000 likes and the top comments were all along the lines of "The victors write history, they will never say the bad guys won WW2" and "Hitler was just fighting the Jewish transgender conspiracy in Germany. Trump will defeat it in America" etc. I used Instagram to follow traditional musicians and people tailoring their own historical clothing. But I can't stay on a platform that is so happy to give a place to that kind of outright hate speech.
- r0fl 2y ago
  I haven’t been served that type of content but I feel the same way as you lately. It feels like it’s shock content with a race to the bottom to get users attention instead of showing interesting content I would actually like
- MathMonkeyMan 2y ago
  Instagram is ad matching machine. Meta is not in the business to care about people.
  - AlecSchueler 2y ago
    It seems short sighted. I know a lot of trans people who use the app, and I used to follow many content creators of colour etc. If we start normalising violently silencing them, paving the way to sending them to camps to work and die, then Meta loses a big part of their customer base.
- MrMcCall 2y ago
  We are all choosing sides, each and every day, between compassionate concern for our fellow human beings, and callous disregard for others' happiness. Choose well, everyone. The karma you reap is always exactly what you deserve, for good or ill. And the pleasure of having power over others is not a good replacement for peace of mind, joy, and happiness.
- jisnsm 2y ago
KaiserPro 2y ago
So what I don't understand is that its perfectly able to categorise this content really well, but is perfectly happy to allow it on the network, even though it clearly violates the community guidelines. Yes moderation at scale is hard but most of the battle is getting decent categorisation.
- r0fl 2y ago
  The extreme content gets the most user interaction and leads to higher daily active users Which is a very important metric. As long as the content is not illegal, if it positively affects time on site/app, it stays for as long as it can on the platforms
  - KaiserPro 2y ago
    > The extreme content gets the most user interaction and leads to higher daily active users For some people yeah, but for a lot, it causes the opposite. That's why its quite heavily policed. Its not like thirst traps which are liberally sprinkled in the first 5 minutes of creating an account.
- jon-wood 2y ago
  The recommendation algorithm isn't necessarily identifying this as shock content or gore, all it's doing is identifying it as having the hallmarks of something that a certain circle of people engage with, followed by it deciding that everyone engages with it and it should be the default. It's possible that internally it's tagging it as gore, it's also possible that it's simply categorising it as "fast moving video with an abundance of red".
  - soco 2y ago
    I would assume after such a successful AI based categorization, one single human moderator could do wonders deciding whether it's an abundance of red or something else. But yeah, why would we censor the money making?
- AlecSchueler 2y ago
r0fl 2y ago
Instagram is borderline a porn/gore website if you open a new account and spend one day doom scrolling The amount of nsfw content including women “breastfeeding” fake babies, wild accidents, weird ultra graphic shock videos is insane. I have a few accounts that I use to post for work. My explore pages are wild compared to my personal account which is mostly tech, cars and architecture I don’t “like” anything on my work accounts, ever, the algorithm just knows what to feed me because of what I click on the most (impossible not to click on those thumbnails)
- milesrout 2y ago
  [flagged]
  - faxmeyourcode 2y ago
    As always, it's the users who are at fault. Nevermind the thousands of kids and teens who are exposed to this shit day in and day out.
    - milesrout 2y ago
      It is the user's fault when the user says in his post that he doesn't interact with the algorithm or give it anything to go on except to click on content he doesn't like and then lo and behold it gives him content he doesn't like. Children should not be on social media at all.
      - faxmeyourcode 2y ago
        So the algorithm should show an elephant stomping a man to death to anyone who starts a new account until they "correct" "their algorithm" by interacting with better content? The issue is that users are being pushed towards violence, gore, and pornography by default. The bar for these giant multi billion dollar tech companies with hundreds of thousands of employees and the most advanced AI tech in the world is truly as low as it's ever been.
    - arkh 2y ago
chewbacha 2y ago
This was the original report from 404media[0]. They are doing really great work over there. [0] https://www.404media.co/instagram-error-turned-reels-into-neverending-scroll-of-murder-gore-and-violence/ https://www.404media.co/instagram-error-turned-reels-into-ne...
- MrMcCall 2y ago
  Seconded. They have been doing great work for a few years now.
- chrisjj 2y ago
  Thanks.
jmclnx 2y ago
Never used it, but how about apologizing for the neo-NAZI items too ? I have to assume since they now allow right-wing propaganda.
donatj 2y ago
Mine has become all things that look like genitalia at a glance but aren't... I'm guessing some algorithm picked up on me stopping on and being like "what the hell?" as liking it...
MrMcCall 2y ago
Mark Zuckerberg will never be happy, after all the horrors his businesses have helped perpetuate, the kinds of lifestyle his brands perpetuate, the folks he has helped put into power. The look on his face at the inauguration is a testament to how awful he feels every minute of every day. Few people in history have caused as much misery to so many people. We ALL reap what we sow, for good or ill, so I suggest doing things that make other people happy, and never cause them misery or grief. Look at the face of Lebron James when he surprised the kids at his personally-funded school in Akron, OH. His smile demonstrates the joy that is possible when one spends their money on others' happiness. "There's still time to change the road you're on." --Stairway to Heaven Happiness requires making money in an ethical way, and then spending it ethically after having earned it.
- djkivi 2y ago
  Lebron also said there ain't no party like a Diddy party.
  - MrMcCall 2y ago
    Lebron had $100M before he was even drafted into the NBA, in 2003. These are two generational world personalities in a utterly uniquely brand-new modern technological era. I saw Diddy's rise in mid-90s radio hip-hop, so by the time Lebron went straight out of high school to the top of the big-time, Diddy was already crazy rich and likely already doing horrible shit. But who knows who knew what and when? So, of course, I'm sure their paths crossed. Always, it's what one does that is weighed. What one sees and perhaps doesn't understand or is kept from knowing, all of which would be difficult to discern at 19 years old or 23 or whatever. You can believe what you want to believe, because everyone does. And everyone makes mistakes, though only a very few make any effort to reevaluate themselves and learn how to be a better person, and then really try.
Jotalea 2y ago
I heard a rumor that certain account was planning a raid on Instagram for today (feb 28 2025) a couple months ago, that said "something will happen on that date". But I just thought it was some kind of weird advertising method from Meta; it kinda worked though, it got me to open the app (I'm too curious).
RodentQueen 2y ago
[dead]
deleted 2y ago
[deleted]
deleted 2y ago
[deleted]
klipklop 2y ago
The most insane death and mutilation content I have ever seen has been just by scrolling my Instagram feed long enough. This stuff is not an accident. No matter how much you say you are not interested a guy in India being ran over by a train will be back within a few minutes of doomscrolling.

70 points by chrisjj 2y ago

> "We have fixed an error that caused some users to see content on their Instagram Reels feed that should not have been recommended" How did that stuff exist on their platform at all?

pbalau 2y ago
Users post it?
- jajko 2y ago
  And nobody really moderates it... pretty sure meta is breaking some laws at least in some jurisdictions but this is nothing new I have in recent year maybe seen a massive increase of various fake posts from unknown groups on facebook. Made up shorts, AI generated fake photos of various weird outrageous claims, which all upon some side check turned out fake but it was not obvious on first glance. I try to report them, mark them as not interesting etc. but they keep popping back without me ever being interested in them. Meta is really shitting on its users and just milking current status, or doesn't care how content on their platforms degrade further, or even quietly allows it. I'd say its the second choice, and it is really by choice. Youtube has similar potential but somehow they manage to sanitize it all well, so clearly its possible.
- brookst 2y ago
  Right? Not a fan of Meta, but the flip side of this is “how dare those censorious corporate overlords limit what I can post to my account”
- theshrike79 2y ago
  Shouldn't a multi-billion dollar company have moderation in place to tag content like that? I'm pretty sure if you start posting spoilers about Marvel properties or straight-up pirated content it'll be taken down as fast as possible. But gore? Nah, why bother.

sdwr 2y ago

If this is because of reduced moderation, does that mean the algorithm defaults to shock content? That gets the most attention, and they're artificially suppressing it most of the time? If the attention economy is a race to the bottom, then the most popular content should be stuff that doesn't quite break the public decency rules.

mc32 2y ago
Ultimately it’s because users consume shock content; if they didn’t it wouldn’t be surfaced and instead they would get whatever generates views be it cats or rainbows.
- gosub100 2y ago
  > users survivorship bias. you aren't account for how many users were repulsed or shocked (read: shook up, not a momentary "wtf") and left the platform.
- cardanome 2y ago
  No, stop blaming the users. Look at this comment: https://news.ycombinator.com/item?id=43204869 https://news.ycombinator.com/item?id=43204869 It clearly shows that passively scrolling on a fresh account will get you gore in no time. Yes, as humans, shock content makes us look but that doesn't mean users want this kind of content. They are being manipulated by these algorithms. Social networks are optimizing for "engagement", meaning ultimately profit, not what users actually want. This is the lazy excuse of every company ever hurting society. Oh, the users want it. No, nobody asked for more gore on Instagram. You are just wanting to save money on moderation.
  - ourmandave 2y ago
    I remember when ISIS posted that beheading video of a journalist, I had to go out of my to avoid seeing it. And that was before Instagram.
  - nullc 2y ago
    I read that comment's "doom scrolling" as implying that it intentionally watched negative-emotion-producing-material. Is this an incorrect reading? If so how does "doom scrolling" differ from "scrolling"?

AlecSchueler 2y ago

Had to delete mine a few weeks ago when I suddenly got a pro-book burning reel, showing Nazis burning books with the caption "They tell you they burned books, bit never which books." It had 30,000 likes and the top comments were all along the lines of "The victors write history, they will never say the bad guys won WW2" and "Hitler was just fighting the Jewish transgender conspiracy in Germany. Trump will defeat it in America" etc. I used Instagram to follow traditional musicians and people tailoring their own historical clothing. But I can't stay on a platform that is so happy to give a place to that kind of outright hate speech.

r0fl 2y ago
I haven’t been served that type of content but I feel the same way as you lately. It feels like it’s shock content with a race to the bottom to get users attention instead of showing interesting content I would actually like
MathMonkeyMan 2y ago
Instagram is ad matching machine. Meta is not in the business to care about people.
- AlecSchueler 2y ago
  It seems short sighted. I know a lot of trans people who use the app, and I used to follow many content creators of colour etc. If we start normalising violently silencing them, paving the way to sending them to camps to work and die, then Meta loses a big part of their customer base.
MrMcCall 2y ago
We are all choosing sides, each and every day, between compassionate concern for our fellow human beings, and callous disregard for others' happiness. Choose well, everyone. The karma you reap is always exactly what you deserve, for good or ill. And the pleasure of having power over others is not a good replacement for peace of mind, joy, and happiness.
jisnsm 2y ago

KaiserPro 2y ago

So what I don't understand is that its perfectly able to categorise this content really well, but is perfectly happy to allow it on the network, even though it clearly violates the community guidelines. Yes moderation at scale is hard but most of the battle is getting decent categorisation.

r0fl 2y ago
The extreme content gets the most user interaction and leads to higher daily active users Which is a very important metric. As long as the content is not illegal, if it positively affects time on site/app, it stays for as long as it can on the platforms
- KaiserPro 2y ago
  > The extreme content gets the most user interaction and leads to higher daily active users For some people yeah, but for a lot, it causes the opposite. That's why its quite heavily policed. Its not like thirst traps which are liberally sprinkled in the first 5 minutes of creating an account.
jon-wood 2y ago
The recommendation algorithm isn't necessarily identifying this as shock content or gore, all it's doing is identifying it as having the hallmarks of something that a certain circle of people engage with, followed by it deciding that everyone engages with it and it should be the default. It's possible that internally it's tagging it as gore, it's also possible that it's simply categorising it as "fast moving video with an abundance of red".
- soco 2y ago
  I would assume after such a successful AI based categorization, one single human moderator could do wonders deciding whether it's an abundance of red or something else. But yeah, why would we censor the money making?
AlecSchueler 2y ago

r0fl 2y ago

Instagram is borderline a porn/gore website if you open a new account and spend one day doom scrolling The amount of nsfw content including women “breastfeeding” fake babies, wild accidents, weird ultra graphic shock videos is insane. I have a few accounts that I use to post for work. My explore pages are wild compared to my personal account which is mostly tech, cars and architecture I don’t “like” anything on my work accounts, ever, the algorithm just knows what to feed me because of what I click on the most (impossible not to click on those thumbnails)

milesrout 2y ago
[flagged]
- faxmeyourcode 2y ago
  As always, it's the users who are at fault. Nevermind the thousands of kids and teens who are exposed to this shit day in and day out.
  - milesrout 2y ago
    It is the user's fault when the user says in his post that he doesn't interact with the algorithm or give it anything to go on except to click on content he doesn't like and then lo and behold it gives him content he doesn't like. Children should not be on social media at all.
    - faxmeyourcode 2y ago
      So the algorithm should show an elephant stomping a man to death to anyone who starts a new account until they "correct" "their algorithm" by interacting with better content? The issue is that users are being pushed towards violence, gore, and pornography by default. The bar for these giant multi billion dollar tech companies with hundreds of thousands of employees and the most advanced AI tech in the world is truly as low as it's ever been.
  - arkh 2y ago

chewbacha 2y ago

This was the original report from 404media[0]. They are doing really great work over there. [0] https://www.404media.co/instagram-error-turned-reels-into-neverending-scroll-of-murder-gore-and-violence/ https://www.404media.co/instagram-error-turned-reels-into-ne...

MrMcCall 2y ago
Seconded. They have been doing great work for a few years now.
chrisjj 2y ago
Thanks.

jmclnx 2y ago

Never used it, but how about apologizing for the neo-NAZI items too ? I have to assume since they now allow right-wing propaganda.

donatj 2y ago

Mine has become all things that look like genitalia at a glance but aren't... I'm guessing some algorithm picked up on me stopping on and being like "what the hell?" as liking it...

MrMcCall 2y ago

Mark Zuckerberg will never be happy, after all the horrors his businesses have helped perpetuate, the kinds of lifestyle his brands perpetuate, the folks he has helped put into power. The look on his face at the inauguration is a testament to how awful he feels every minute of every day. Few people in history have caused as much misery to so many people. We ALL reap what we sow, for good or ill, so I suggest doing things that make other people happy, and never cause them misery or grief. Look at the face of Lebron James when he surprised the kids at his personally-funded school in Akron, OH. His smile demonstrates the joy that is possible when one spends their money on others' happiness. "There's still time to change the road you're on." --Stairway to Heaven Happiness requires making money in an ethical way, and then spending it ethically after having earned it.

djkivi 2y ago
Lebron also said there ain't no party like a Diddy party.
- MrMcCall 2y ago
  Lebron had $100M before he was even drafted into the NBA, in 2003. These are two generational world personalities in a utterly uniquely brand-new modern technological era. I saw Diddy's rise in mid-90s radio hip-hop, so by the time Lebron went straight out of high school to the top of the big-time, Diddy was already crazy rich and likely already doing horrible shit. But who knows who knew what and when? So, of course, I'm sure their paths crossed. Always, it's what one does that is weighed. What one sees and perhaps doesn't understand or is kept from knowing, all of which would be difficult to discern at 19 years old or 23 or whatever. You can believe what you want to believe, because everyone does. And everyone makes mistakes, though only a very few make any effort to reevaluate themselves and learn how to be a better person, and then really try.

Jotalea 2y ago

I heard a rumor that certain account was planning a raid on Instagram for today (feb 28 2025) a couple months ago, that said "something will happen on that date". But I just thought it was some kind of weird advertising method from Meta; it kinda worked though, it got me to open the app (I'm too curious).

RodentQueen 2y ago

[dead]

deleted 2y ago

[deleted]

deleted 2y ago

[deleted]

klipklop 2y ago

The most insane death and mutilation content I have ever seen has been just by scrolling my Instagram feed long enough. This stuff is not an accident. No matter how much you say you are not interested a guy in India being ran over by a train will be back within a few minutes of doomscrolling.