3 ms·

This is the core of the problem that we're facing on every modern "all the things" platform. How do you create a general content site, without spending money o

This is the core of the problem that we're facing on every modern "all the things" platform. How do you create a general content site, without spending money on human review and curation of all uploads, while segregating erotic and non-erotic content, without having your entire site banned/blocked/gated around the world for hosting erotic content? This has affected Amazon, Blogspot, Facebook, Google, Imgur, Patreon, Reddit, Tumblr, Twitch, Youtube. Every general content site has this problem. Most try to solve it with "user flags", which are promptly weaponized and abused to silence others. Problems constantly recur around statues, breastfeeding, swimsuits. Advertisers want content to be segmented with certainty, but site operators don't want to pay human beings to do that. Standards of "erotic" vary wildly per city, country, religion. The list goes on. What if human curation is the only valid solution? We haven't found any other solution that works yet, and AI is barely able to recognize human beings, much less discern whether content is intended to be categorized as erotic, non-erotic, or both.

robto 5y ago
Do you think this moderation paradox - of needing curation at scale, but automatic moderation + "user flag" moderation being inadequate - is a universal problem, or do you think it's specific to ad-funded platforms? I wonder if a patron-driven community that sets its own moderation policy wouldn't have this problem, because when a topic gets too large it will inevitably splinter into smaller communities where the preferred moderation style of each subset can be catered to.
- floatingatoll 5y ago
  Mm, dunno. I suspect it's a universal problem, given that Patreon also fights with "how erotic is too erotic". But the core of today's practical concerns might simply be that advertising doesn't pay enough to afford content moderation.
  - deleted 5y ago
    [deleted]
dangerface 5y ago
I agree ai curation doesn't work but I don't think human curation works much better. Twitch has a team of human curators that will ban channels, no one knows who this team is there is no trial or opportunity to defend your self, channel or content. The person judging may be in a bad mood today, often diffrent people will get diffrent ban lengths for the same thing, there is no consistency or review process.
- floatingatoll 5y ago
  Disagreeing with the decisions of moderators is orthogonal to comparing AI and Human moderation. AI moderators are terrible at enforcing a complex set of practices and rules. Human moderators are excellent at enforcing a complex set of practices and rules. Both are subject to the whims of those defining the practices and rules, and their enforcement will always fall within scope of those definitions. If Twitch’s moderators are given full empowerment to set the rules they moderate by, then it is appropriate to blame them for what is perceived as faulty moderation decisions. Chances are, they do not have any such authority, and at best can “recommend” or “advise” while being compelled to silence by their overlords. You are absolutely right that humans make biased decisions, no matter how much we work to correct for it; judge sentencing becomes more strict when they’re hungry or their local sports team lost, and referees are less likely to penalize the home team when in the home stadium with home fans yelling at them. But these biases can be documented, studied, and gradually accounted for in training and post-decision reviews. It’s not perfect, but it’s still great. With AI moderation, we bake the biases into the dataset and training models, and then we throw complex rule making decisions at a nascent ML network that can’t discern the simplest of decisions that any human moderators could: “Is this erotic or non-erotic content?”. Setting aside the biases of any possible training set versus Rule 34, it is dead simple to outwit any AI in this regard, in ways that most human moderators would detect instantly. It’s not perfect, and in fact it’s pretty awful. I agree that the rulemaking decisions of modern sites are often terrible, biased, and exclusionary of entire categories of human beings and behaviors. However, the errors made by human moderators trying to service the complexity assigned to them do not somehow excuse the staggering naivety and incompetence of AI moderators. The heavyweights in this Ml moderation industry — Google, Amazon, Facebook — have failed to deliver effective ML moderation after billions of dollars and cumulative decades of investment, and paper over their failure by using user flags and human review teams to disguise the sharp edges of their solutions.
deleted 5y ago
[deleted]

robto 5y ago

Do you think this moderation paradox - of needing curation at scale, but automatic moderation + "user flag" moderation being inadequate - is a universal problem, or do you think it's specific to ad-funded platforms? I wonder if a patron-driven community that sets its own moderation policy wouldn't have this problem, because when a topic gets too large it will inevitably splinter into smaller communities where the preferred moderation style of each subset can be catered to.

floatingatoll 5y ago
Mm, dunno. I suspect it's a universal problem, given that Patreon also fights with "how erotic is too erotic". But the core of today's practical concerns might simply be that advertising doesn't pay enough to afford content moderation.
- deleted 5y ago
  [deleted]

dangerface 5y ago

I agree ai curation doesn't work but I don't think human curation works much better. Twitch has a team of human curators that will ban channels, no one knows who this team is there is no trial or opportunity to defend your self, channel or content. The person judging may be in a bad mood today, often diffrent people will get diffrent ban lengths for the same thing, there is no consistency or review process.

floatingatoll 5y ago
Disagreeing with the decisions of moderators is orthogonal to comparing AI and Human moderation. AI moderators are terrible at enforcing a complex set of practices and rules. Human moderators are excellent at enforcing a complex set of practices and rules. Both are subject to the whims of those defining the practices and rules, and their enforcement will always fall within scope of those definitions. If Twitch’s moderators are given full empowerment to set the rules they moderate by, then it is appropriate to blame them for what is perceived as faulty moderation decisions. Chances are, they do not have any such authority, and at best can “recommend” or “advise” while being compelled to silence by their overlords. You are absolutely right that humans make biased decisions, no matter how much we work to correct for it; judge sentencing becomes more strict when they’re hungry or their local sports team lost, and referees are less likely to penalize the home team when in the home stadium with home fans yelling at them. But these biases can be documented, studied, and gradually accounted for in training and post-decision reviews. It’s not perfect, but it’s still great. With AI moderation, we bake the biases into the dataset and training models, and then we throw complex rule making decisions at a nascent ML network that can’t discern the simplest of decisions that any human moderators could: “Is this erotic or non-erotic content?”. Setting aside the biases of any possible training set versus Rule 34, it is dead simple to outwit any AI in this regard, in ways that most human moderators would detect instantly. It’s not perfect, and in fact it’s pretty awful. I agree that the rulemaking decisions of modern sites are often terrible, biased, and exclusionary of entire categories of human beings and behaviors. However, the errors made by human moderators trying to service the complexity assigned to them do not somehow excuse the staggering naivety and incompetence of AI moderators. The heavyweights in this Ml moderation industry — Google, Amazon, Facebook — have failed to deliver effective ML moderation after billions of dollars and cumulative decades of investment, and paper over their failure by using user flags and human review teams to disguise the sharp edges of their solutions.

deleted 5y ago

[deleted]