4 ms·

Its clear that the broken element in this process is not the AI banning the writer of the post, but the poor presumably human review

atemerev 4y ago
Good human reviews are, well, nearly impossible at the scale of Twitter or Google. It would require employing hundreds of thousands of reviewers, training them, and paying them decent salaries for one of the most horrible jobs in the world. So, everybody is trying to automate as much as possible in this area. I am sure that in this case, if there is a human even involved in the escalation, it takes them a few seconds to decide on each case, with poor decision quality.
- crote 4y ago
  Your point being? If you are building a many-billion-dollar company on user-submitted content, moderation is pretty much a necessity. A hospital can't decide not to sterilize because it is "too difficult" either, can it? Especially when it comes to Google, which tries to entangle itself in every aspect of your life, being banned from all services without any potential for a proper review just because a neural network decided to throw a hissy fit is completely unacceptable.
- Kiro 4y ago
  Funny how you get downvoted for saying something everyone on HN agreed on when "Open Letter from Facebook Content Moderators" was published, with comments like "That is a simple problem for which Facebook should be able to apply existing technology. There is a lot Facebook can do, but they are not, because they are lazy.", calling for more automation and less traumatic moderation work.
  - simoncion 4y ago
    I didn't agree on it. What likely happened was a bunch of excited folks wrote a whole bunch of words, and that volume gave the appearance of consensus. Automated review is not good enough for services that are so large that they are de facto public squares. If the world were just, when you got that big, the rules would change to prohibit you from doing service-wide content filtering. > ...calling for more automation and less traumatic moderation work. Pay me 80% of what I would be making as a programmer (and keep up with changes in the market rate for programmers), and I'd be totally willing to do that moderation work. [0] There are _tons_ of people who have my intestinal fortitude out there... problem is that the enormous "social media" companies don't want to pay enough to get people who are well-suited for the work. [0] You might counter with "Oh, well you can't _possibly_ imagine the things you'll see, you're all bluster and bravado and will quit within the month!". I'd counter with "Yeah, well you have no idea the things that I _have_ seen.".
- Wowfunhappy 4y ago
  It's the year 2032, and Wings, Jack Dorsey's self-driving taxi service, is a hit. Because Wings doesn't have to pay human drivers, most of their rides are free. The company makes money by showing ads during rides and selling data about users's movement habits. Wings is not profitable yet, but revenue is through the roof, and the company is expected to IPO for a zillion dollars. Unfortunately, Wings's self-driving AI is much likely to destroy property than a human driver. As a result, some members of the public have called for every Wings car to be remotely controlled by a human. However, others push back. Employing good human drivers would be nearly impossible at the scale of Wings. Here is an interview with a poor grandmother who relies on Wings to see her family, and here's how a nonprofit uses Wings to find homes for lost puppies. You don't really don't want to go back to paid Uber and Lyft rides, do you? There are a couple of problems with this analogy. But the core point is this—if a company can't sustain a key aspect of their business at scale, then perhaps they shouldn't be able to scale.
  - atemerev 4y ago
    There is an equilibrium point beyond which mass protests and regulations pressure start. Every company is iterating and optimizing to find this equilibrium point, and then stays slightly behind it.
    - Wowfunhappy 4y ago
      I happen to think today's social media companies are well past the equilibrium point.
      - atemerev 4y ago
        Well, I don't see mass protests against Google and Facebook yet, so perhaps they are not. (I do see mass protests against Uber though).
- danaris 4y ago
  Oh, so they'd have to spend a bunch of money to treat people like human beings? Boo fucking hoo. The money is the problem here. We should require them to spend that money, and if they can't remain profitable while employing sufficient human moderators to not be a net detriment to society, they should not be allowed to keep operating. We've gotten way too far down the rabbit hole of "any company that can conceivably exist must be allowed to exist, so long as they can make a profit." We really need to get back to companies having to provide some meaningful net benefit to society, rather than just making scads of money for their owners.
  - atemerev 4y ago
    OK, press for regulation! So far, I haven’t seen any mass action that would be sufficient for the big tech to change their ways. People are mostly content with the status quo.
spicyusername 4y ago
I'm not so sure I agree. The typical argument for AI moderation in these situations is something along the lines of "Twitter is so big how could we possibly moderate at this scale without AI?" Does a company have a right to scale faster than it has the ability to properly moderate? Should they? Looking at a more extreme example: If General Motors used AI to design cars and they increased the likelihood of fatalities, would we feel any different?
- belltaco 4y ago
  >Does a company have a right to scale faster than it has the ability to properly moderate? A right? Yes. >Looking at a more extreme example: If General Motors used AI to design cars and they increased the likelihood of fatalities, would we feel any different? Getting (temporarily?) banned from Twitter isn't even close to being as bad as a fatality, so the GM example doesn't make any sense.
czx4f4bd 4y ago
An AI arbitrarily banning someone for a completely innocuous post isn't broken?
- sascha_sl 4y ago
  If the false positive rate is low and human review is swift and not broken, I really wouldn't mind. It's a force multiplier that every social media platform needs to not drown in costs for human moderation. Especially human moderation as described above. Unless you get people to pay for social media. Good luck with that.
- jhanschoo 4y ago
  > arbitrarily banning If that ban by AI is available for review and appeal, it's just triage that admits false positives. The alternatives that I can fathom is to either allow unsavory and potentially harmful content to be present on your platform until human eyes get on them, or to have all comments await moderation until human eyes get on them.
tux3 4y ago
Human review is pretense, it costs more to do anything else. A human is told to process these as fast as possible, so the issue can be resolved. With these incentives, it wouldn't be reasonable to actually watch the videos in question. That's not what the system is designed to reward. The human is there to rubber stamp the AI decision as fast as possible, so that Twitter can tell you your case has been reviewed by a human and settled. The important metric is speed, because that directly translates into cost.
tyingq 4y ago
It's telling that the AI doesn't just flag something, without banning it, and let a human do a proper review.

atemerev 4y ago

Good human reviews are, well, nearly impossible at the scale of Twitter or Google. It would require employing hundreds of thousands of reviewers, training them, and paying them decent salaries for one of the most horrible jobs in the world. So, everybody is trying to automate as much as possible in this area. I am sure that in this case, if there is a human even involved in the escalation, it takes them a few seconds to decide on each case, with poor decision quality.

crote 4y ago
Your point being? If you are building a many-billion-dollar company on user-submitted content, moderation is pretty much a necessity. A hospital can't decide not to sterilize because it is "too difficult" either, can it? Especially when it comes to Google, which tries to entangle itself in every aspect of your life, being banned from all services without any potential for a proper review just because a neural network decided to throw a hissy fit is completely unacceptable.
Kiro 4y ago
Funny how you get downvoted for saying something everyone on HN agreed on when "Open Letter from Facebook Content Moderators" was published, with comments like "That is a simple problem for which Facebook should be able to apply existing technology. There is a lot Facebook can do, but they are not, because they are lazy.", calling for more automation and less traumatic moderation work.
- simoncion 4y ago
  I didn't agree on it. What likely happened was a bunch of excited folks wrote a whole bunch of words, and that volume gave the appearance of consensus. Automated review is not good enough for services that are so large that they are de facto public squares. If the world were just, when you got that big, the rules would change to prohibit you from doing service-wide content filtering. > ...calling for more automation and less traumatic moderation work. Pay me 80% of what I would be making as a programmer (and keep up with changes in the market rate for programmers), and I'd be totally willing to do that moderation work. [0] There are _tons_ of people who have my intestinal fortitude out there... problem is that the enormous "social media" companies don't want to pay enough to get people who are well-suited for the work. [0] You might counter with "Oh, well you can't _possibly_ imagine the things you'll see, you're all bluster and bravado and will quit within the month!". I'd counter with "Yeah, well you have no idea the things that I _have_ seen.".
Wowfunhappy 4y ago
It's the year 2032, and Wings, Jack Dorsey's self-driving taxi service, is a hit. Because Wings doesn't have to pay human drivers, most of their rides are free. The company makes money by showing ads during rides and selling data about users's movement habits. Wings is not profitable yet, but revenue is through the roof, and the company is expected to IPO for a zillion dollars. Unfortunately, Wings's self-driving AI is much likely to destroy property than a human driver. As a result, some members of the public have called for every Wings car to be remotely controlled by a human. However, others push back. Employing good human drivers would be nearly impossible at the scale of Wings. Here is an interview with a poor grandmother who relies on Wings to see her family, and here's how a nonprofit uses Wings to find homes for lost puppies. You don't really don't want to go back to paid Uber and Lyft rides, do you? There are a couple of problems with this analogy. But the core point is this—if a company can't sustain a key aspect of their business at scale, then perhaps they shouldn't be able to scale.
- atemerev 4y ago
  There is an equilibrium point beyond which mass protests and regulations pressure start. Every company is iterating and optimizing to find this equilibrium point, and then stays slightly behind it.
  - Wowfunhappy 4y ago
    I happen to think today's social media companies are well past the equilibrium point.
    - atemerev 4y ago
      Well, I don't see mass protests against Google and Facebook yet, so perhaps they are not. (I do see mass protests against Uber though).
danaris 4y ago
Oh, so they'd have to spend a bunch of money to treat people like human beings? Boo fucking hoo. The money is the problem here. We should require them to spend that money, and if they can't remain profitable while employing sufficient human moderators to not be a net detriment to society, they should not be allowed to keep operating. We've gotten way too far down the rabbit hole of "any company that can conceivably exist must be allowed to exist, so long as they can make a profit." We really need to get back to companies having to provide some meaningful net benefit to society, rather than just making scads of money for their owners.
- atemerev 4y ago
  OK, press for regulation! So far, I haven’t seen any mass action that would be sufficient for the big tech to change their ways. People are mostly content with the status quo.

spicyusername 4y ago

I'm not so sure I agree. The typical argument for AI moderation in these situations is something along the lines of "Twitter is so big how could we possibly moderate at this scale without AI?" Does a company have a right to scale faster than it has the ability to properly moderate? Should they? Looking at a more extreme example: If General Motors used AI to design cars and they increased the likelihood of fatalities, would we feel any different?

belltaco 4y ago
>Does a company have a right to scale faster than it has the ability to properly moderate? A right? Yes. >Looking at a more extreme example: If General Motors used AI to design cars and they increased the likelihood of fatalities, would we feel any different? Getting (temporarily?) banned from Twitter isn't even close to being as bad as a fatality, so the GM example doesn't make any sense.

czx4f4bd 4y ago

An AI arbitrarily banning someone for a completely innocuous post isn't broken?

sascha_sl 4y ago
If the false positive rate is low and human review is swift and not broken, I really wouldn't mind. It's a force multiplier that every social media platform needs to not drown in costs for human moderation. Especially human moderation as described above. Unless you get people to pay for social media. Good luck with that.
jhanschoo 4y ago
> arbitrarily banning If that ban by AI is available for review and appeal, it's just triage that admits false positives. The alternatives that I can fathom is to either allow unsavory and potentially harmful content to be present on your platform until human eyes get on them, or to have all comments await moderation until human eyes get on them.

tux3 4y ago

Human review is pretense, it costs more to do anything else. A human is told to process these as fast as possible, so the issue can be resolved. With these incentives, it wouldn't be reasonable to actually watch the videos in question. That's not what the system is designed to reward. The human is there to rubber stamp the AI decision as fast as possible, so that Twitter can tell you your case has been reviewed by a human and settled. The important metric is speed, because that directly translates into cost.

tyingq 4y ago

It's telling that the AI doesn't just flag something, without banning it, and let a human do a proper review.