4 ms·

What are you trying to accomplish with this comment? The intractability of the overall problem is a topic of interest, sure. But this thread is pointing out ho

What are you trying to accomplish with this comment? The intractability of the overall problem is a topic of interest, sure. But this thread is pointing out how poor Google’s current implementation/attempt to solve that problem seems to be. The parent comment adds to the dialog by pointing out that while innocent content is getting removed, truly harmful content still exists and is rampant. Is it piling on? Maybe. But Google is earning the criticism. This doesn’t imply that the solution is simple, but highlights how broken things are. Google is not the only place people can post user generated content, but they have a growing reputation for handling these issues more poorly than most, which isn’t just dismissible because it’s a hard problem to solve.

dredmorbius 5y ago
Before you read the rest of this comment: how many channels do you expect that YouTube removes in year, or a day, or an hour? My goal is to try to inject some awareness or concept of scale and accuracy being demanded. You've had the opportunity to do so, you've neglected that option, instead doubling down on the emotive argument. The data happen to be readily available online for anyone bothering to look. The problem with both the original comment and your reply here is the denominator is ignored: - How many accounts exist? - How many are removed? - How many of those removals are appealed? I happened to look up the numbers for Facebook a few days ago when similar complaints were lodged against them. And again: I believe in large part that the power and influence of these platforms requires an extraordinary level of care. That said, Facebook blocked 99.8% of fake accounts preemptively before they were flagged by user, in 2021Q3. For a total of 1.8 billion accounts removed. Whatever experience ordinary users of the system had of fake accounts was after that removal of 1.8 billion accounts, nearly 8 billion per year. https://transparency.fb.com/data/community-standards-enforcement/fake-accounts/facebook/ https://transparency.fb.com/data/community-standards-enforce... I'm looking up Google's own transparency report based on your comment. The YouTube Community Standards report is here: https://transparencyreport.google.com/youtube-policy/removals https://transparencyreport.google.com/youtube-policy/removal... The headline number is that in 2021Q3, YouTube removed 2.4 million channels. That's an annualised rate of nearly 20 million, and the quarterly removal rate roughly doubled from the 2020Q4 & 2021Q1 rate begining in 2021Q2. The cadence is 55,000 channels per day, or about 2,300 per hour. For comparison, manual reviewers at maximum capacity can moderation about 700--800 items in a day, leaving roughly 30 seconds per item, based on the New York Times comment moderation desk. http://www.nytimes.com/interactive/2016/09/20/insider/approve-or-reject-moderation-quiz.html?_r=1 http://www.nytimes.com/interactive/2016/09/20/insider/approv... I've discussed content moderation in terms to total personal media capacity a few years ago here: https://old.reddit.com/r/dredmorbius/comments/53qbx3/estimating_personal_daily_media_consumption/ https://old.reddit.com/r/dredmorbius/comments/53qbx3/estimat... https://old.reddit.com/r/dredmorbius/comments/7qya12/information_security_peps_podesta_who_are_you/ https://old.reddit.com/r/dredmorbius/comments/7qya12/informa... Note that we're taking both FB and YT accounts at face value, that the removals were legitimate. Facebook offers information on appeals, Google (so far as I can tell) does not. But at a first approximation the removals seem to be mostly valid. A huge problem is that a small fraction of a very large number ... is still a large number. A 1% misclassification rate of 20 million channels is 200,000 channels/year. A 99.99% accurate classification is still 2,000 channels year. For Facebook, with 5 billion removals/year, a 99.99% accurate rate is 500,000 erroniously banned profiles. And the risk profiles are asymmetric. For the platform. 99.99% success is ... well, pretty good. But for each individual who's erroneously classified, the consequences range from a major annoyance to catastrophic. And lest you think I'm making the case that this is "good enough" or "acceptable losses", I'm not. I think it would be fairly straightforward to make a strong argument for the heavy regulation or break-up of these services strictly on this basis. They have too much control and influence, both at the macro and micro scales, and even near-perfect operations, management, and policies are not nearly good enough. You might also want to think about your own actions, and those of formal and informal institutions in which you believe and/or rely, and consider what their own failure rates in judgement and adjudication are. How many false arrests, or evictions, or debt collections, or sentencings, or executions are you willing to live with, transacted in the name of society?
- haswell 5y ago
  First, there's some great information in this comment. Thanks for taking the time to elaborate. However, we're on very different wavelengths and having conversations about two very different things. > My goal is to try to inject some awareness or concept of scale and accuracy being demanded. You've had the opportunity to do so, you've neglected that option, instead doubling down on the emotive argument. You're doing so by highjacking a thread that is unrelated to your end goal. I've neglected nothing, because the point I was trying to make is unrelated. You seem to believe that all exploration of the problem (someone loses access to their account due to bad algorithms taking enforcement actions) must be looked at through the lens of how hard the problem is to solve, or the discussion isn't worth having. It's fine to raise awareness about the implications of scale and accuracy, and that's an interesting topic to examine, but you've gone a few steps too far and seem to believe that one must look at the user impact through this lens, or they shouldn't look at all. Treat the problem of moderation as a black box for a moment: 1) Content goes in, moderated content comes out with some error rate 2) Based on the moderation result, some action is taken against the content and potentially the user 3) Due to the inherent error rate, incorrect actions will be taken against some content/users And assume for a moment that the error rate cannot be improved; it's still possible to have a meaningful conversation about #2 and #3 without delving into the seeming impossibility of changing the error rate in #1. There are myriad angles to explore ranging from: how to best run a manual review program, to how to improve enforcement actions so they have less impact on legitimate users, to improving the appeals process to ensure it doesn't take a HN front page article to get something done. Someone could choose not to even attempt improving the error rate, or discover that there are diminishing returns to lowering the error rate at some point. These are tangentially related, but still technically orthogonal to improving the enforcement/appeals process. Of course it's still completely valid to examine #1 directly! There's plenty to explore: reducing false positive rates in algorithms, optimizing the manual review process, etc. But it's silly to cast aspersions on the basis that someone chose to focus on the customer facing problem without acknowledging how hard it is to fix that problem. It's just as reasonable to talk about how dangerous "self driving" vehicles are today. Of course it's a monumentally difficult problem to solve. But we still need to have meaningful outcome-focused conversations about these cars despite those complexities. > You might also want to think about your own actions, and those of formal and informal institutions in which you believe and/or rely I do often think about such things, but today's discussion is about a common issue plaguing the YouTube content ecosystem. There are appropriate places and threads to discuss those topics. I'm mostly aligned with what you're saying, and do understand how difficult these problems are to solve. I take issue with the idea that folks are somehow involved in some kind of dereliction of duty or low value discussion for not injecting sympathy/awe at how difficult the problem is to solve.

dredmorbius 5y ago

Before you read the rest of this comment: how many channels do you expect that YouTube removes in year, or a day, or an hour? My goal is to try to inject some awareness or concept of scale and accuracy being demanded. You've had the opportunity to do so, you've neglected that option, instead doubling down on the emotive argument. The data happen to be readily available online for anyone bothering to look. The problem with both the original comment and your reply here is the denominator is ignored: - How many accounts exist? - How many are removed? - How many of those removals are appealed? I happened to look up the numbers for Facebook a few days ago when similar complaints were lodged against them. And again: I believe in large part that the power and influence of these platforms requires an extraordinary level of care. That said, Facebook blocked 99.8% of fake accounts preemptively before they were flagged by user, in 2021Q3. For a total of 1.8 billion accounts removed. Whatever experience ordinary users of the system had of fake accounts was after that removal of 1.8 billion accounts, nearly 8 billion per year. https://transparency.fb.com/data/community-standards-enforcement/fake-accounts/facebook/ https://transparency.fb.com/data/community-standards-enforce... I'm looking up Google's own transparency report based on your comment. The YouTube Community Standards report is here: https://transparencyreport.google.com/youtube-policy/removals https://transparencyreport.google.com/youtube-policy/removal... The headline number is that in 2021Q3, YouTube removed 2.4 million channels. That's an annualised rate of nearly 20 million, and the quarterly removal rate roughly doubled from the 2020Q4 & 2021Q1 rate begining in 2021Q2. The cadence is 55,000 channels per day, or about 2,300 per hour. For comparison, manual reviewers at maximum capacity can moderation about 700--800 items in a day, leaving roughly 30 seconds per item, based on the New York Times comment moderation desk. http://www.nytimes.com/interactive/2016/09/20/insider/approve-or-reject-moderation-quiz.html?_r=1 http://www.nytimes.com/interactive/2016/09/20/insider/approv... I've discussed content moderation in terms to total personal media capacity a few years ago here: https://old.reddit.com/r/dredmorbius/comments/53qbx3/estimating_personal_daily_media_consumption/ https://old.reddit.com/r/dredmorbius/comments/53qbx3/estimat... https://old.reddit.com/r/dredmorbius/comments/7qya12/information_security_peps_podesta_who_are_you/ https://old.reddit.com/r/dredmorbius/comments/7qya12/informa... Note that we're taking both FB and YT accounts at face value, that the removals were legitimate. Facebook offers information on appeals, Google (so far as I can tell) does not. But at a first approximation the removals seem to be mostly valid. A huge problem is that a small fraction of a very large number ... is still a large number. A 1% misclassification rate of 20 million channels is 200,000 channels/year. A 99.99% accurate classification is still 2,000 channels year. For Facebook, with 5 billion removals/year, a 99.99% accurate rate is 500,000 erroniously banned profiles. And the risk profiles are asymmetric. For the platform. 99.99% success is ... well, pretty good. But for each individual who's erroneously classified, the consequences range from a major annoyance to catastrophic. And lest you think I'm making the case that this is "good enough" or "acceptable losses", I'm not. I think it would be fairly straightforward to make a strong argument for the heavy regulation or break-up of these services strictly on this basis. They have too much control and influence, both at the macro and micro scales, and even near-perfect operations, management, and policies are not nearly good enough. You might also want to think about your own actions, and those of formal and informal institutions in which you believe and/or rely, and consider what their own failure rates in judgement and adjudication are. How many false arrests, or evictions, or debt collections, or sentencings, or executions are you willing to live with, transacted in the name of society?

haswell 5y ago
First, there's some great information in this comment. Thanks for taking the time to elaborate. However, we're on very different wavelengths and having conversations about two very different things. > My goal is to try to inject some awareness or concept of scale and accuracy being demanded. You've had the opportunity to do so, you've neglected that option, instead doubling down on the emotive argument. You're doing so by highjacking a thread that is unrelated to your end goal. I've neglected nothing, because the point I was trying to make is unrelated. You seem to believe that all exploration of the problem (someone loses access to their account due to bad algorithms taking enforcement actions) must be looked at through the lens of how hard the problem is to solve, or the discussion isn't worth having. It's fine to raise awareness about the implications of scale and accuracy, and that's an interesting topic to examine, but you've gone a few steps too far and seem to believe that one must look at the user impact through this lens, or they shouldn't look at all. Treat the problem of moderation as a black box for a moment: 1) Content goes in, moderated content comes out with some error rate 2) Based on the moderation result, some action is taken against the content and potentially the user 3) Due to the inherent error rate, incorrect actions will be taken against some content/users And assume for a moment that the error rate cannot be improved; it's still possible to have a meaningful conversation about #2 and #3 without delving into the seeming impossibility of changing the error rate in #1. There are myriad angles to explore ranging from: how to best run a manual review program, to how to improve enforcement actions so they have less impact on legitimate users, to improving the appeals process to ensure it doesn't take a HN front page article to get something done. Someone could choose not to even attempt improving the error rate, or discover that there are diminishing returns to lowering the error rate at some point. These are tangentially related, but still technically orthogonal to improving the enforcement/appeals process. Of course it's still completely valid to examine #1 directly! There's plenty to explore: reducing false positive rates in algorithms, optimizing the manual review process, etc. But it's silly to cast aspersions on the basis that someone chose to focus on the customer facing problem without acknowledging how hard it is to fix that problem. It's just as reasonable to talk about how dangerous "self driving" vehicles are today. Of course it's a monumentally difficult problem to solve. But we still need to have meaningful outcome-focused conversations about these cars despite those complexities. > You might also want to think about your own actions, and those of formal and informal institutions in which you believe and/or rely I do often think about such things, but today's discussion is about a common issue plaguing the YouTube content ecosystem. There are appropriate places and threads to discuss those topics. I'm mostly aligned with what you're saying, and do understand how difficult these problems are to solve. I take issue with the idea that folks are somehow involved in some kind of dereliction of duty or low value discussion for not injecting sympathy/awe at how difficult the problem is to solve.