5 ms·

Why would someone expect the recommendation algorithm to really know anything regarding whether videos violate their own policies? If the platform already knew

Why would someone expect the recommendation algorithm to really know anything regarding whether videos violate their own policies? If the platform already knew the videos violated their policies, presumably they'd already be blocked/removed in the first place. It's not magic.

talaketu 5y ago
One might expect a publisher to pay particular attention to the content they promote.
- arthurcolle 5y ago
  Too bad they exist in a quantum superposition of publisher and platform states simultaneously, while also being neither. Pretty nice gig if you can get it!
  - Nasrudith 5y ago
    Platform and publisher are meaningless terms born of latter day propaganda. They are about as meaningful legally as asking if they are a unicorn or a leprechaun.
d3nj4l 5y ago
I think you're missing the point. According to TFA, the YouTube algorithm likes content that breaks their policies very much: Recommended videos were 40% times more likely to be regretted than videos searched for. In 43.6% of cases where Mozilla had data about videos a volunteer watched before a regret, the recommendation was completely unrelated to the previous videos that the volunteer watched. YouTube Regrets tend to perform extremely well on the platform, with reported videos acquiring 70% more views per day than other videos watched by volunteers. So it's not that the algorithm is just "blind" to content that breaks their policies, it seems to prefer them over other content.
- PragmaticPulp 5y ago
  Reading the report, this wasn't actually a random sampling of typical users. They used opt-in reporting from a self-selected group of volunteers who were interested in reporting "regrettable" YouTube recommendations. If some of these users were deliberately scouring YouTube for objectionable content, it follows that their recommendations would start matching similarly objectionable content for them. After reading the report, I don't see how any of this is representative of a typical user's YouTube experience. It certainly doesn't resemble my own experience at all.
- randallsquared 5y ago
  > it seems to prefer them over other content. Isn't it likely that it prefers them because they're more likely to be fully watched/upvoted/commented on/otherwise engaging? Presumably, that fact together with objections to the content are the reason to have a policy...
- philwelch 5y ago
  There are some pretty obvious explanations to all of these points: * People don't search for content they don't want to watch. * Videos that a specific person doesn't want to watch are often unrelated to videos that the same person does want to watch. * If a video receives a high number of views, that is probably a signal to the algorithm that the video is popular enough to recommend more often. If anything, those numbers are lower than what I would expect.
MereInterest 5y ago
Say I go over to Fred, and ask him for a book recommendation. I've known Fred for a while, and he's given me good book recommendations in the past, so I usually listen to what he says. Except this time, Fred says that the book I'm most likely to enjoy is Mein Kampf. I'm flabbergasted, because this doesn't sound like the Fred I know. Fred says that he has no idea what the book is about, but there's a small number of people who vigorously recommend it, and they're just so gosh-darned enthusiastic about it that now he's recommending it to anybody with even the slightest amount of interest in politics. He knows that I just read Elie Wiesel's Night, so it seemed right up my alley to Fred. Swap out "Youtube" for "Fred", and maybe that helps to show the issue. I'm not making an exaggeration here. Youtube has recommended that I watch Nazi propaganda, because I had just recently watched videos debunking those exact propaganda films. When "recommendations" become "recommendation engines", and the human steps out of the loop, the recommender still has a moral responsibility for the things they recommend. Without a human in the loop, it's a lot easier to close one's eyes to that responsibility. Maybe the solution is to go back to things that don't scale, and keep humans in the loop. Maybe the solution is to spend just as much effort on the quality of the recommendations as the addictiveness of the recommendations. But the current state is not sustainable.
- root_axis 5y ago
  I think your analogy supports the GP's point. The recommendation engine doesn't understand that the recommendation is a violation, it makes a heuristic error because there is an indomitable mountain of content to classify and it lacks the sophistication to accurately do so. It's certainly logical to hold YouTube accountable for the content it recommends, but it's also naive to expect it to achieve the impossible.
- colejohnson66 5y ago
  > Fred says that he has no idea what the book is about… And that’s the thing people don’t understand when they blame YouTube. If Fred is YouTube, and Fred doesn’t know that Mein Kampf is propaganda, how is YouTube supposed to remove it? You’re just reconfirming OP’s point: YouTube doesn’t know that a video violates its policies.
mayneack 5y ago
Sure, that's plausible, but it seems equally likely that Youtube _can_ detect videos that violate their policy and yet chooses to wait until they get some threshold of reports/media attention because those videos are also very engaging. Having the policy allows them to react when they get pressure without impacting the bottom line.
- asdfasgasdgasdg 5y ago
  I wouldn't say equally likely. They both have a non-zero probability of being true but a pattern of malfeasance such as you describe would be detected pretty quickly. Imagine this or thousands of similar conversations: "Brad, have you noticed that our porn videos get removed only when they get over a certain view count?" "I had noticed that, Bob! I wonder if that observation would be interesting to the media?!" It also has minimal upside because YouTube makes more money on high view count videos by far, so removing only when a certain level of fame has hit a video cuts off the part that is economically interesting to YouTube. The tail is a loss leader. Your scenario is not very plausible to me.
  - mayneack 5y ago
    "Brad, we can't deploy your conspiracy theory classifier because it would reduce the number of high value viewer engagements by 2.7%. Let's just stick with the version that depends on accumulating reports. It's not a big deal because we can always intervene if something embarrassing slips through."
    - asdfasgasdgasdg 5y ago
      I'm unaware of any evidence consistent with this version of how things work there. By all accounts, YouTube are unlenient with bans and blacklistings. Can you please cite a source?
      - mayneack 5y ago
        The original comment I was replying to was speculating about how they imagined youtube worked without evidence. All I did was provide an alternate explanation. I wouldn't even go so far as to suggest it's a bad thing that social networks avoid using more aggressive classifiers, just that I'm skeptical of the "we don't have the ability" explanation. By all account YouTube are arbitrary with bans and blockings. That's not the same as unlenient. The standard example of the phenomenon I'm describing is from Twitter. They successfully keep nazi content out of German twitter, but not the rest of the world. That suggests that Twitter's moderation is partially constrained by their own policies, not just technical ability. I don't think this is a far fetched understanding of how any social network works. https://www.snopes.com/fact-check/twitter-germany-nazis/ https://www.snopes.com/fact-check/twitter-germany-nazis/
PragmaticPulp 5y ago
> Why would someone expect the recommendation algorithm to really know anything regarding whether videos violate their own policies? There's a big push to blame recommendation engines in the wake of social media misinformation about elections and COVID-19. There's a growing perception, verging on conspiracy theory, that big social media companies are deliberately tuning their recommendation engines to promote this misinformation while publicly claiming to do the opposite. This Mozilla report feels like a strangely clumsy attack piece. They relied on self-selected volunteers to proactively report videos that they called "regrets" using a Mozilla plugin. Users were given little direction about what to report as a "regret" but according to the Mozilla report they marked as many as 1 in 8 videos as "should not be on YouTube". I have no idea how these self-reporters were going down a YouTube recommendations rabbit hole that ended up with 1 in 8 recommendations being policy-breaking or inappropriate, but that couldn't be farther from my own experience. I stream YouTube in the background while working on projects around the house and I can't recall the last time I was recommended a strange video that was somehow policy-breaking. The YouTube algorithm is clearly very self-reinforcing. I suppose if the volunteers were actively searching YouTube for "regret" videos that broke policy then it's likely that the recommendation engine would be triggered into serving up more, similar videos. That's obviously not a valid representation of the typical user experience, though.
- ABCLAW 5y ago
  >I stream YouTube in the background while working on projects around the house and I can't recall the last time I was recommended a strange video that was somehow policy-breaking. I have a fairly curated stream of videos due to years of Youtube use. I never see objectionable content in my recommended segment. I mostly have videos related to my career area and a few creators who focus on games I play. However, I've logged in on other computers, or via VPNs on different devices and found that if I just started with a random folksy video (let's say a bandsaw restoration), I would be recommended objectionable (overtly anti-black or anti-Semitic material) very rapidly. What's more, this random walk would then seed my recommended videos with a high proportion of similar videos. I'm not sure how much directed viewing would need to occur to re-adjust the recommendations. This was such a large issue that at certain offices I worked at, I'd pre-seed my youtube history with a few 'chill cafe background music' videos first to prevent a re-occurrence of the time when I walked away from my computer after logging onto a new terminal and opening up what I figured was some decent background noise, then coming back to a pro-hitler historical documentary. This may be because of my location, demographic, activity on the network or the selection of initial videos. My experience is nothing more than an anecdote, but it leads me to think that extreme content does well with respect to engagement, and accordingly is promoted by an engagement focused process. Is this correct? I don't know. I just know it isn't far fetched.

talaketu 5y ago

One might expect a publisher to pay particular attention to the content they promote.

arthurcolle 5y ago
Too bad they exist in a quantum superposition of publisher and platform states simultaneously, while also being neither. Pretty nice gig if you can get it!
- Nasrudith 5y ago
  Platform and publisher are meaningless terms born of latter day propaganda. They are about as meaningful legally as asking if they are a unicorn or a leprechaun.

d3nj4l 5y ago

I think you're missing the point. According to TFA, the YouTube algorithm likes content that breaks their policies very much: Recommended videos were 40% times more likely to be regretted than videos searched for. In 43.6% of cases where Mozilla had data about videos a volunteer watched before a regret, the recommendation was completely unrelated to the previous videos that the volunteer watched. YouTube Regrets tend to perform extremely well on the platform, with reported videos acquiring 70% more views per day than other videos watched by volunteers. So it's not that the algorithm is just "blind" to content that breaks their policies, it seems to prefer them over other content.

PragmaticPulp 5y ago
Reading the report, this wasn't actually a random sampling of typical users. They used opt-in reporting from a self-selected group of volunteers who were interested in reporting "regrettable" YouTube recommendations. If some of these users were deliberately scouring YouTube for objectionable content, it follows that their recommendations would start matching similarly objectionable content for them. After reading the report, I don't see how any of this is representative of a typical user's YouTube experience. It certainly doesn't resemble my own experience at all.
randallsquared 5y ago
> it seems to prefer them over other content. Isn't it likely that it prefers them because they're more likely to be fully watched/upvoted/commented on/otherwise engaging? Presumably, that fact together with objections to the content are the reason to have a policy...
philwelch 5y ago
There are some pretty obvious explanations to all of these points: * People don't search for content they don't want to watch. * Videos that a specific person doesn't want to watch are often unrelated to videos that the same person does want to watch. * If a video receives a high number of views, that is probably a signal to the algorithm that the video is popular enough to recommend more often. If anything, those numbers are lower than what I would expect.

MereInterest 5y ago

Say I go over to Fred, and ask him for a book recommendation. I've known Fred for a while, and he's given me good book recommendations in the past, so I usually listen to what he says. Except this time, Fred says that the book I'm most likely to enjoy is Mein Kampf. I'm flabbergasted, because this doesn't sound like the Fred I know. Fred says that he has no idea what the book is about, but there's a small number of people who vigorously recommend it, and they're just so gosh-darned enthusiastic about it that now he's recommending it to anybody with even the slightest amount of interest in politics. He knows that I just read Elie Wiesel's Night, so it seemed right up my alley to Fred. Swap out "Youtube" for "Fred", and maybe that helps to show the issue. I'm not making an exaggeration here. Youtube has recommended that I watch Nazi propaganda, because I had just recently watched videos debunking those exact propaganda films. When "recommendations" become "recommendation engines", and the human steps out of the loop, the recommender still has a moral responsibility for the things they recommend. Without a human in the loop, it's a lot easier to close one's eyes to that responsibility. Maybe the solution is to go back to things that don't scale, and keep humans in the loop. Maybe the solution is to spend just as much effort on the quality of the recommendations as the addictiveness of the recommendations. But the current state is not sustainable.

root_axis 5y ago
I think your analogy supports the GP's point. The recommendation engine doesn't understand that the recommendation is a violation, it makes a heuristic error because there is an indomitable mountain of content to classify and it lacks the sophistication to accurately do so. It's certainly logical to hold YouTube accountable for the content it recommends, but it's also naive to expect it to achieve the impossible.
colejohnson66 5y ago
> Fred says that he has no idea what the book is about… And that’s the thing people don’t understand when they blame YouTube. If Fred is YouTube, and Fred doesn’t know that Mein Kampf is propaganda, how is YouTube supposed to remove it? You’re just reconfirming OP’s point: YouTube doesn’t know that a video violates its policies.

mayneack 5y ago

Sure, that's plausible, but it seems equally likely that Youtube _can_ detect videos that violate their policy and yet chooses to wait until they get some threshold of reports/media attention because those videos are also very engaging. Having the policy allows them to react when they get pressure without impacting the bottom line.

asdfasgasdgasdg 5y ago
I wouldn't say equally likely. They both have a non-zero probability of being true but a pattern of malfeasance such as you describe would be detected pretty quickly. Imagine this or thousands of similar conversations: "Brad, have you noticed that our porn videos get removed only when they get over a certain view count?" "I had noticed that, Bob! I wonder if that observation would be interesting to the media?!" It also has minimal upside because YouTube makes more money on high view count videos by far, so removing only when a certain level of fame has hit a video cuts off the part that is economically interesting to YouTube. The tail is a loss leader. Your scenario is not very plausible to me.
- mayneack 5y ago
  "Brad, we can't deploy your conspiracy theory classifier because it would reduce the number of high value viewer engagements by 2.7%. Let's just stick with the version that depends on accumulating reports. It's not a big deal because we can always intervene if something embarrassing slips through."
  - asdfasgasdgasdg 5y ago
    I'm unaware of any evidence consistent with this version of how things work there. By all accounts, YouTube are unlenient with bans and blacklistings. Can you please cite a source?
    - mayneack 5y ago
      The original comment I was replying to was speculating about how they imagined youtube worked without evidence. All I did was provide an alternate explanation. I wouldn't even go so far as to suggest it's a bad thing that social networks avoid using more aggressive classifiers, just that I'm skeptical of the "we don't have the ability" explanation. By all account YouTube are arbitrary with bans and blockings. That's not the same as unlenient. The standard example of the phenomenon I'm describing is from Twitter. They successfully keep nazi content out of German twitter, but not the rest of the world. That suggests that Twitter's moderation is partially constrained by their own policies, not just technical ability. I don't think this is a far fetched understanding of how any social network works. https://www.snopes.com/fact-check/twitter-germany-nazis/ https://www.snopes.com/fact-check/twitter-germany-nazis/

PragmaticPulp 5y ago

> Why would someone expect the recommendation algorithm to really know anything regarding whether videos violate their own policies? There's a big push to blame recommendation engines in the wake of social media misinformation about elections and COVID-19. There's a growing perception, verging on conspiracy theory, that big social media companies are deliberately tuning their recommendation engines to promote this misinformation while publicly claiming to do the opposite. This Mozilla report feels like a strangely clumsy attack piece. They relied on self-selected volunteers to proactively report videos that they called "regrets" using a Mozilla plugin. Users were given little direction about what to report as a "regret" but according to the Mozilla report they marked as many as 1 in 8 videos as "should not be on YouTube". I have no idea how these self-reporters were going down a YouTube recommendations rabbit hole that ended up with 1 in 8 recommendations being policy-breaking or inappropriate, but that couldn't be farther from my own experience. I stream YouTube in the background while working on projects around the house and I can't recall the last time I was recommended a strange video that was somehow policy-breaking. The YouTube algorithm is clearly very self-reinforcing. I suppose if the volunteers were actively searching YouTube for "regret" videos that broke policy then it's likely that the recommendation engine would be triggered into serving up more, similar videos. That's obviously not a valid representation of the typical user experience, though.

ABCLAW 5y ago
>I stream YouTube in the background while working on projects around the house and I can't recall the last time I was recommended a strange video that was somehow policy-breaking. I have a fairly curated stream of videos due to years of Youtube use. I never see objectionable content in my recommended segment. I mostly have videos related to my career area and a few creators who focus on games I play. However, I've logged in on other computers, or via VPNs on different devices and found that if I just started with a random folksy video (let's say a bandsaw restoration), I would be recommended objectionable (overtly anti-black or anti-Semitic material) very rapidly. What's more, this random walk would then seed my recommended videos with a high proportion of similar videos. I'm not sure how much directed viewing would need to occur to re-adjust the recommendations. This was such a large issue that at certain offices I worked at, I'd pre-seed my youtube history with a few 'chill cafe background music' videos first to prevent a re-occurrence of the time when I walked away from my computer after logging onto a new terminal and opening up what I figured was some decent background noise, then coming back to a pro-hitler historical documentary. This may be because of my location, demographic, activity on the network or the selection of initial videos. My experience is nothing more than an anecdote, but it leads me to think that extreme content does well with respect to engagement, and accordingly is promoted by an engagement focused process. Is this correct? I don't know. I just know it isn't far fetched.