5 ms·

I'm not sure how this proves the algorithm is doing a poor job. Any NSFW classifier is going to have a non-trivial error rate. Tumblr hosts LOTS of images. So i

I'm not sure how this proves the algorithm is doing a poor job. Any NSFW classifier is going to have a non-trivial error rate. Tumblr hosts LOTS of images. So if you have a non-trivial error rate and multiply it by LOTS of images you are going to get LOTS of images being misclassified incorrectly. It's not surprising someone was able to write a story about a bunch of images being misclassified. This doesn't seem to tell us anything about whether the algorithm has a high error rate or not.

dandare 8y ago
You have my upvote, but at the same time, Tumblr doesn't know this? What is their communication strategy for taking the wind off of people and articles making fun of them?
netcan 8y ago
If you want to take a fatalistic view of it... go a certain step further. multiply it by LOTS of images you are going to get LOTS of images being misclassified and lots of complaints/mockery. Also context, Tumblr users are being censored in ways they weren't previously and the censor is an error prone machine. This isn't the same thing as implementing nsfw filters to Google search.
int_19h 8y ago
In this case, the ultimate arbiters are the users of the service. Judging by their feedback so far, it is doing a poor job (and it doesn't matter if it's state of the art etc - that just means that our state of the art algorithms are that bad).
- kkarakk 8y ago
  tumblr's announcemnt about removing nsfw content got flagged,i'd say the algo is trash
BlackLotus89 8y ago
Since we don't have any real stats we can't classify how well the algorithm is doing, one important thought though is how the algorithm is designed. If you got a good filtering method you can either optimize for a false positive rate or a false negative rate. Let's compare it to a simple spam filter (because if you look at it objectively that's all it is) would you rather have spam in your inbox or loose mails to your spam filter? Spam filter (normally) prioritize a high false negative so you have less spam, but don't loose content. Having a filter that obviously has a high false positive rate is just bad for the content creators (on which tumblr depends)
opencl 8y ago
Maybe it's a good classifier by some measure but the error rate is clearly way too high for indiscriminately deleting everything it detects to be a good idea, and that is exactly what Tumblr plans on doing in 2 weeks. Randomly deleting posts that in no way violate the rules is going to piss a lot of people off even if "only" 5% of posts are affected. This is kind of an inherent problem with automated moderation tools. And not new to Tumblr either, there is some sort of mysterious spam filter applied to the ask system that results in a lot of legitimate messages not being sent that people have complained about for ages. And maybe my dash is not a representative sample but there really does seem to be a huge number of misclassified posts.
pbhjpbhj 8y ago
I wonder how successful demanding self-classification would be? I'd imagine it would be better than an AI, and you could have user-reporting, AI monitoring, random employee sampling, SERPs monitoring to catch those who purposefully misclassified and apply the ban-hammer.

dandare 8y ago

You have my upvote, but at the same time, Tumblr doesn't know this? What is their communication strategy for taking the wind off of people and articles making fun of them?

netcan 8y ago

If you want to take a fatalistic view of it... go a certain step further. multiply it by LOTS of images you are going to get LOTS of images being misclassified and lots of complaints/mockery. Also context, Tumblr users are being censored in ways they weren't previously and the censor is an error prone machine. This isn't the same thing as implementing nsfw filters to Google search.

int_19h 8y ago

In this case, the ultimate arbiters are the users of the service. Judging by their feedback so far, it is doing a poor job (and it doesn't matter if it's state of the art etc - that just means that our state of the art algorithms are that bad).

kkarakk 8y ago
tumblr's announcemnt about removing nsfw content got flagged,i'd say the algo is trash

BlackLotus89 8y ago

Since we don't have any real stats we can't classify how well the algorithm is doing, one important thought though is how the algorithm is designed. If you got a good filtering method you can either optimize for a false positive rate or a false negative rate. Let's compare it to a simple spam filter (because if you look at it objectively that's all it is) would you rather have spam in your inbox or loose mails to your spam filter? Spam filter (normally) prioritize a high false negative so you have less spam, but don't loose content. Having a filter that obviously has a high false positive rate is just bad for the content creators (on which tumblr depends)

opencl 8y ago

Maybe it's a good classifier by some measure but the error rate is clearly way too high for indiscriminately deleting everything it detects to be a good idea, and that is exactly what Tumblr plans on doing in 2 weeks. Randomly deleting posts that in no way violate the rules is going to piss a lot of people off even if "only" 5% of posts are affected. This is kind of an inherent problem with automated moderation tools. And not new to Tumblr either, there is some sort of mysterious spam filter applied to the ask system that results in a lot of legitimate messages not being sent that people have complained about for ages. And maybe my dash is not a representative sample but there really does seem to be a huge number of misclassified posts.

pbhjpbhj 8y ago

I wonder how successful demanding self-classification would be? I'd imagine it would be better than an AI, and you could have user-reporting, AI monitoring, random employee sampling, SERPs monitoring to catch those who purposefully misclassified and apply the ban-hammer.