5 ms·

If this is because of reduced moderation, does that mean the algorithm defaults to shock content? That gets the most attention, and they're artificially suppres

by sdwr 2y ago

If this is because of reduced moderation, does that mean the algorithm defaults to shock content? That gets the most attention, and they're artificially suppressing it most of the time? If the attention economy is a race to the bottom, then the most popular content should be stuff that doesn't quite break the public decency rules.

mc32 2y ago
Ultimately it’s because users consume shock content; if they didn’t it wouldn’t be surfaced and instead they would get whatever generates views be it cats or rainbows.
- gosub100 2y ago
  > users survivorship bias. you aren't account for how many users were repulsed or shocked (read: shook up, not a momentary "wtf") and left the platform.
- cardanome 2y ago
  No, stop blaming the users. Look at this comment: https://news.ycombinator.com/item?id=43204869 https://news.ycombinator.com/item?id=43204869 It clearly shows that passively scrolling on a fresh account will get you gore in no time. Yes, as humans, shock content makes us look but that doesn't mean users want this kind of content. They are being manipulated by these algorithms. Social networks are optimizing for "engagement", meaning ultimately profit, not what users actually want. This is the lazy excuse of every company ever hurting society. Oh, the users want it. No, nobody asked for more gore on Instagram. You are just wanting to save money on moderation.
  - ourmandave 2y ago
    I remember when ISIS posted that beheading video of a journalist, I had to go out of my to avoid seeing it. And that was before Instagram.
  - nullc 2y ago
    I read that comment's "doom scrolling" as implying that it intentionally watched negative-emotion-producing-material. Is this an incorrect reading? If so how does "doom scrolling" differ from "scrolling"?
    - droopyEyelids 2y ago
      I want to answer your question, but i don’t know what “negative-emotion-producing-material” means, nor how to look up definitions. Could you type out an explanation by hand for me?
      - fragmede 2y ago
        Stuff that makes you feel bad; about the state of the world or just life. Helpless, powerless. A story about how orphans died because a billionaire bought the building and kicked then out. Somebody's pet dying. A big corporation steam rolling little people. A friend getting cancer. War.
        droopyEyelids 2y ago
        OK, doom scrolling is “scrolling compulsively” without intention, and with the vague feeling that time would be better spent elsewhere. It doesn’t have to do with the content.
    - rcxdude 2y ago
      I generally interpret "doomscrolling" as "compulsive scrolling", e.g. whem someone is bored and just keeps going through one of the infinitly scrolling social media feeds.
      - AlecSchueler 2y ago
        Bored or tired or just "filling in time" at e.g. a bus stop. That is, times when your energy isn't completely there for making sure you do everything right to manage your micro-behaviours (things like not hovering too long or reading too many comments on a post) in order to curate your feed.
  - brookst 2y ago
    It’s somewhat more nuanced than that; a classic sampling problem. When Instagram knows nothing about user X, it trends to shocking content that user X may not want. Which is bad. But the reason it does so is that the shocking content has higher engagement, in average, among users A - W. It is not right to blame user X, and it is very likely that users D, H, and M are dismayed at the shock content but can’t help slowing down when they scroll past it. So we can blame Meta to some degree, and users A - W (except D, H, and M) to some degree, and users D, H, and M to some degree. But if the total ad revenue from users A-W was lower when shock content was featured, user X would not be presented with it. Of course, Meta could just be decent. But so could the other users. The fact that neither are true is an emergent property of sociopathic corporations and an unfortunate side of human nature.
  - Arnt 2y ago
    There are two things here. The company has an algorithm that tried to find out what makes users click/scroll onwards/return soon. And then there is what users like. The company chooses to act as though the two are equal. That's not given. Nobody forces Insta to assume that.
    - MrMcCall 2y ago
      No one is ever forced to be evil. Everything in the human world is the result of someone's choices. "Shit rolls downhill." And all of Zuck's works are a detriment to humanity.
  - intended 2y ago
    What did you mean stop blaming the users? You mean that the firms are incentivized to show outrage inducing content to gain attention? If so, that’s also NOT the problem with firms. You, me, and everyone with a phone is in some form of information and attention war. Or over farming situation. - There is a competitive process for attention - What is crazy today is boring tomorrow, as people get desensitized - all content competes with all other content, for the little working memory that people have. Personally I describe it as a gearing ratio issue. We have one small gear which is everyone’s short term attention. In the mean time, we have a larger gear which is the content cycle. This gear is spinning faster and faster as the velocity of content generation increases. Anyone who takes a stand to slow down or make their content less attractive, will get out-competed by more competitive content. Any moderation of this process breaks the lay definition of free speech. So you have a runaway effect where the content must get more outrageous. This isn’t the fault of anyone, but a fundamental problem when free speech meets faster and faster content creation tools. We’re at the point where GenAI slop will simply out compete everything, and grow everywhere like a weed. I’m telling people I know that you have to give up on facts, the signal to noise ratio of the future is going to be hopelessly poor. Instead, you will have to make do with whatever content you are looking at, and focus on the techniques which you use to interact with the content or people. Its hard to blame a singular firm for this, when anyone who takes a stand will be crushed by the wheel of competition and free-speech.
    - cardanome 2y ago
      Competition? The social network market is split between a few big monopolists that will buy up or destroy any new upstart. And even if we assume a free market, I don't follow your logic. Twitter has vastly more lax moderation since Elon Musk took over and people are LEAVING it. It clearly shows that many users actually prefer better moderated platforms.
      - intended 2y ago
        The underlying competition I am referring to is the competition for attention. —— Yet, if we are too look at firms, TikTok came on the scene and at everyone’s lunch, till they came up with clones. Your example, Bluesky, is also competition. ——- The fact that people are leaving Nazis doesn’t mean that people are somehow escaping the battle for attention. See for example, the radicalization of the left which has finally accelerated enough to be its own force. Given enough time, Bluesky will also find that its users must deal with the same forces. If you want content to succeed on Bsky, you need to get eyeballs, which means you need to compete with other things people will watch. —— You also missed the part that moderation construes a violation of the popular definition of “Free speech”, and is thus censorship. I know it’s nonsense. I used to run a boring community; people eventually leave or start importing drama from the more active parts of the web. You have to tamp down on that, which results in the “violation of free speech”. So you now have pressures to stop moderation. Heck, Wikipedia is being targeted, senators are bankrupting misinformation researchers, all to ensure that they are able to maintain the lowest level of moderation possible. People need to wise up to the point of free speech, i.e. ensuring the functioning of a market place of ideas.
  - neuroticnews25 2y ago
    "passively scrolling" is still sending a stream of user actions and choices. But you're right, I trained myself to swipe past animal cruelty and ragebait on FB as fast as possible and still get it recommended. Maybe in some indirect way making me feel bad still increases my engagement.
- xtiansimon 2y ago
  Which users most heavily weight algorithms for viral effects? The guy who looks at dogs and bikes, or the teen user who doesn’t know their arse from their elbow?
- rightbyte 2y ago
  I had this experience where Facebook.com started to spam me more and more car accident videos. Probably since I hovered over them to block the sender. I would guess that there could be a similar effect here. And I guess my eyes looks longer on a car getting stuck on a rail road crossing than something that is not inheritly dangerous, too.
- Tade0 2y ago
  It only surfaces because the algorithm is set to a simple more engagement = more visibility. If it was removing outliers, then this would not only not be the case, but also arguably create a more pleasant experience for everyone. Of course greed prevails, which is why we are where we are.
- UncleMeat 2y ago
  It is hard to look away from somebody beating a dog with a belt. In our daily lives we are ethically trained not to look away from this sort of thing. Even just pausing your scrolling over a video is enough to train your feed to show you more videos like that. So when you encounter something like animal cruelty in your feed and you, quite naturally, pause for a moment in shock you will start seeing more of this content. Again, as you pause in shock on this content you'll keep seeing more. And more. And we don't permit this "it is because users consume this content" as a general rule. Users consume hardcore porn. Nevertheless, these platforms seek to not show people hardcore porn.
micromacrofoot 2y ago
it certainly means they can successfully detect it, which makes it all the more abhorrent that they don't moderate it very well (or at all)
drpossum 2y ago
> If the attention economy is a race to the bottom, then the most popular content should be stuff that doesn't quite break the public decency rules. I'm not sure I follow if that's indeed what gets people's attention and engagement, good or bad.

by sdwr 2y ago

mc32 2y ago

Ultimately it’s because users consume shock content; if they didn’t it wouldn’t be surfaced and instead they would get whatever generates views be it cats or rainbows.

gosub100 2y ago
> users survivorship bias. you aren't account for how many users were repulsed or shocked (read: shook up, not a momentary "wtf") and left the platform.
cardanome 2y ago
No, stop blaming the users. Look at this comment: https://news.ycombinator.com/item?id=43204869 https://news.ycombinator.com/item?id=43204869 It clearly shows that passively scrolling on a fresh account will get you gore in no time. Yes, as humans, shock content makes us look but that doesn't mean users want this kind of content. They are being manipulated by these algorithms. Social networks are optimizing for "engagement", meaning ultimately profit, not what users actually want. This is the lazy excuse of every company ever hurting society. Oh, the users want it. No, nobody asked for more gore on Instagram. You are just wanting to save money on moderation.
- ourmandave 2y ago
  I remember when ISIS posted that beheading video of a journalist, I had to go out of my to avoid seeing it. And that was before Instagram.
- nullc 2y ago
  I read that comment's "doom scrolling" as implying that it intentionally watched negative-emotion-producing-material. Is this an incorrect reading? If so how does "doom scrolling" differ from "scrolling"?
  - droopyEyelids 2y ago
    I want to answer your question, but i don’t know what “negative-emotion-producing-material” means, nor how to look up definitions. Could you type out an explanation by hand for me?
    - fragmede 2y ago
      Stuff that makes you feel bad; about the state of the world or just life. Helpless, powerless. A story about how orphans died because a billionaire bought the building and kicked then out. Somebody's pet dying. A big corporation steam rolling little people. A friend getting cancer. War.
      - droopyEyelids 2y ago
        OK, doom scrolling is “scrolling compulsively” without intention, and with the vague feeling that time would be better spent elsewhere. It doesn’t have to do with the content.
  - rcxdude 2y ago
    I generally interpret "doomscrolling" as "compulsive scrolling", e.g. whem someone is bored and just keeps going through one of the infinitly scrolling social media feeds.
    - AlecSchueler 2y ago
      Bored or tired or just "filling in time" at e.g. a bus stop. That is, times when your energy isn't completely there for making sure you do everything right to manage your micro-behaviours (things like not hovering too long or reading too many comments on a post) in order to curate your feed.
- brookst 2y ago
  It’s somewhat more nuanced than that; a classic sampling problem. When Instagram knows nothing about user X, it trends to shocking content that user X may not want. Which is bad. But the reason it does so is that the shocking content has higher engagement, in average, among users A - W. It is not right to blame user X, and it is very likely that users D, H, and M are dismayed at the shock content but can’t help slowing down when they scroll past it. So we can blame Meta to some degree, and users A - W (except D, H, and M) to some degree, and users D, H, and M to some degree. But if the total ad revenue from users A-W was lower when shock content was featured, user X would not be presented with it. Of course, Meta could just be decent. But so could the other users. The fact that neither are true is an emergent property of sociopathic corporations and an unfortunate side of human nature.
- Arnt 2y ago
  There are two things here. The company has an algorithm that tried to find out what makes users click/scroll onwards/return soon. And then there is what users like. The company chooses to act as though the two are equal. That's not given. Nobody forces Insta to assume that.
  - MrMcCall 2y ago
    No one is ever forced to be evil. Everything in the human world is the result of someone's choices. "Shit rolls downhill." And all of Zuck's works are a detriment to humanity.
- intended 2y ago
  What did you mean stop blaming the users? You mean that the firms are incentivized to show outrage inducing content to gain attention? If so, that’s also NOT the problem with firms. You, me, and everyone with a phone is in some form of information and attention war. Or over farming situation. - There is a competitive process for attention - What is crazy today is boring tomorrow, as people get desensitized - all content competes with all other content, for the little working memory that people have. Personally I describe it as a gearing ratio issue. We have one small gear which is everyone’s short term attention. In the mean time, we have a larger gear which is the content cycle. This gear is spinning faster and faster as the velocity of content generation increases. Anyone who takes a stand to slow down or make their content less attractive, will get out-competed by more competitive content. Any moderation of this process breaks the lay definition of free speech. So you have a runaway effect where the content must get more outrageous. This isn’t the fault of anyone, but a fundamental problem when free speech meets faster and faster content creation tools. We’re at the point where GenAI slop will simply out compete everything, and grow everywhere like a weed. I’m telling people I know that you have to give up on facts, the signal to noise ratio of the future is going to be hopelessly poor. Instead, you will have to make do with whatever content you are looking at, and focus on the techniques which you use to interact with the content or people. Its hard to blame a singular firm for this, when anyone who takes a stand will be crushed by the wheel of competition and free-speech.
  - cardanome 2y ago
    Competition? The social network market is split between a few big monopolists that will buy up or destroy any new upstart. And even if we assume a free market, I don't follow your logic. Twitter has vastly more lax moderation since Elon Musk took over and people are LEAVING it. It clearly shows that many users actually prefer better moderated platforms.
    - intended 2y ago
      The underlying competition I am referring to is the competition for attention. —— Yet, if we are too look at firms, TikTok came on the scene and at everyone’s lunch, till they came up with clones. Your example, Bluesky, is also competition. ——- The fact that people are leaving Nazis doesn’t mean that people are somehow escaping the battle for attention. See for example, the radicalization of the left which has finally accelerated enough to be its own force. Given enough time, Bluesky will also find that its users must deal with the same forces. If you want content to succeed on Bsky, you need to get eyeballs, which means you need to compete with other things people will watch. —— You also missed the part that moderation construes a violation of the popular definition of “Free speech”, and is thus censorship. I know it’s nonsense. I used to run a boring community; people eventually leave or start importing drama from the more active parts of the web. You have to tamp down on that, which results in the “violation of free speech”. So you now have pressures to stop moderation. Heck, Wikipedia is being targeted, senators are bankrupting misinformation researchers, all to ensure that they are able to maintain the lowest level of moderation possible. People need to wise up to the point of free speech, i.e. ensuring the functioning of a market place of ideas.
- neuroticnews25 2y ago
  "passively scrolling" is still sending a stream of user actions and choices. But you're right, I trained myself to swipe past animal cruelty and ragebait on FB as fast as possible and still get it recommended. Maybe in some indirect way making me feel bad still increases my engagement.
xtiansimon 2y ago
Which users most heavily weight algorithms for viral effects? The guy who looks at dogs and bikes, or the teen user who doesn’t know their arse from their elbow?
rightbyte 2y ago
I had this experience where Facebook.com started to spam me more and more car accident videos. Probably since I hovered over them to block the sender. I would guess that there could be a similar effect here. And I guess my eyes looks longer on a car getting stuck on a rail road crossing than something that is not inheritly dangerous, too.
Tade0 2y ago
It only surfaces because the algorithm is set to a simple more engagement = more visibility. If it was removing outliers, then this would not only not be the case, but also arguably create a more pleasant experience for everyone. Of course greed prevails, which is why we are where we are.
UncleMeat 2y ago
It is hard to look away from somebody beating a dog with a belt. In our daily lives we are ethically trained not to look away from this sort of thing. Even just pausing your scrolling over a video is enough to train your feed to show you more videos like that. So when you encounter something like animal cruelty in your feed and you, quite naturally, pause for a moment in shock you will start seeing more of this content. Again, as you pause in shock on this content you'll keep seeing more. And more. And we don't permit this "it is because users consume this content" as a general rule. Users consume hardcore porn. Nevertheless, these platforms seek to not show people hardcore porn.

micromacrofoot 2y ago

it certainly means they can successfully detect it, which makes it all the more abhorrent that they don't moderate it very well (or at all)

drpossum 2y ago

> If the attention economy is a race to the bottom, then the most popular content should be stuff that doesn't quite break the public decency rules. I'm not sure I follow if that's indeed what gets people's attention and engagement, good or bad.