5 ms·

Discord banned 8k users for posting benign grid images

33 points by logickkk1 3mo ago

Ravi4649 3mo ago
Imagine you suddenly getting banned after posting a picture of a chessboard
- DonHopkins 3mo ago
  Or X11 default root weave pattern. https://matttproud.com/blog/posts/x-window-system-boot-stipple.html https://matttproud.com/blog/posts/x-window-system-boot-stipp...
- nitwit005 3mo ago
  It's an old problem: https://gizmodo.com/british-cops-want-to-use-ai-to-spot-porn-but-it-keeps-m-1821384511 https://gizmodo.com/british-cops-want-to-use-ai-to-spot-porn... I've been fairly convinced a lot of social media bans, where people are genuinely confused about the bans, are just automatic moderation problems.
  - Hugsbox 3mo ago
    It's for detecting porn? I genuinely don't get it. I regularly post actual nudes on discord and never so much as a warning, crazy that a chessboard or grids are doing it. I wonder what makes an algorithm view a grid pattern as pornographic.
    - wmf 3mo ago
      Somebody said people were posting images of QR codes that contained (links to?) CSAM. But instead of using a QR code decoder, it sounds like Discord used perceptual hashing or AI to detect and ban all QR-like images.
      - Hugsbox 3mo ago
        Ohhh that makes a lot more sense. Well, their implementation doesn't, but it makes sense that grids were mistaken for QR codes.
novia 3mo ago
> “When our staff reviewed and cleared those accounts, the same bug prevented the ban from being lifted automatically, so it just stayed in place,” Discord says. It seems like a similar issue is affecting Twitter currently. People are being suspended for inauthentic behavior, they appeal, they get an email saying the appeal was accepted, and then, nothing, the account doesn't come back. Is there perhaps a third party moderation platform behind both of these services? https://www.reddit.com/r/twitterhelp/comments/1sr7t1m/x_restored_my_account_no_violation_but_im_still/ https://www.reddit.com/r/twitterhelp/comments/1sr7t1m/x_rest... https://devcommunity.x.com/t/restoration-emails-for-suspended-accounts-inauthentic-behaviors-are-false-accounts-do-not-get-restored-suspension-is-not-lifted-as-email-indicates/268394 https://devcommunity.x.com/t/restoration-emails-for-suspende...
Terr_ 3mo ago
Optimistic future: Moderation scaling issues prevent monopolies and ensure people have alternatives. Pessimistic future: There will be a couple major services and some people will be frozen-out victims of a system focused on cheap rather than correct.

33 points by logickkk1 3mo ago

Ravi4649 3mo ago

Imagine you suddenly getting banned after posting a picture of a chessboard

DonHopkins 3mo ago
Or X11 default root weave pattern. https://matttproud.com/blog/posts/x-window-system-boot-stipple.html https://matttproud.com/blog/posts/x-window-system-boot-stipp...
nitwit005 3mo ago
It's an old problem: https://gizmodo.com/british-cops-want-to-use-ai-to-spot-porn-but-it-keeps-m-1821384511 https://gizmodo.com/british-cops-want-to-use-ai-to-spot-porn... I've been fairly convinced a lot of social media bans, where people are genuinely confused about the bans, are just automatic moderation problems.
- Hugsbox 3mo ago
  It's for detecting porn? I genuinely don't get it. I regularly post actual nudes on discord and never so much as a warning, crazy that a chessboard or grids are doing it. I wonder what makes an algorithm view a grid pattern as pornographic.
  - wmf 3mo ago
    Somebody said people were posting images of QR codes that contained (links to?) CSAM. But instead of using a QR code decoder, it sounds like Discord used perceptual hashing or AI to detect and ban all QR-like images.
    - Hugsbox 3mo ago
      Ohhh that makes a lot more sense. Well, their implementation doesn't, but it makes sense that grids were mistaken for QR codes.

novia 3mo ago

> “When our staff reviewed and cleared those accounts, the same bug prevented the ban from being lifted automatically, so it just stayed in place,” Discord says. It seems like a similar issue is affecting Twitter currently. People are being suspended for inauthentic behavior, they appeal, they get an email saying the appeal was accepted, and then, nothing, the account doesn't come back. Is there perhaps a third party moderation platform behind both of these services? https://www.reddit.com/r/twitterhelp/comments/1sr7t1m/x_restored_my_account_no_violation_but_im_still/ https://www.reddit.com/r/twitterhelp/comments/1sr7t1m/x_rest... https://devcommunity.x.com/t/restoration-emails-for-suspended-accounts-inauthentic-behaviors-are-false-accounts-do-not-get-restored-suspension-is-not-lifted-as-email-indicates/268394 https://devcommunity.x.com/t/restoration-emails-for-suspende...

Terr_ 3mo ago

Optimistic future: Moderation scaling issues prevent monopolies and ensure people have alternatives. Pessimistic future: There will be a couple major services and some people will be frozen-out victims of a system focused on cheap rather than correct.