4 ms·

It would be quite humorous, and also quite telling of the experience of those implementing this, if it turned out to be classic class imbalance during training.

by dpwm 8y ago

It would be quite humorous, and also quite telling of the experience of those implementing this, if it turned out to be classic class imbalance during training. If 90% of the training dataset is nsfw, and you train a classifier naively on that dataset, and your test and validation data is 90% nsfw, then 90% accuracy is very easy.

newen 8y ago
Oh god...I remember my first project I just ran an ML algorithm against the dataset, got high accuracy, boasted to colleagues, and then explored the dataset only to realize it was highly unbalanced. So embarrassing.
AJ007 8y ago
I would really, really like to know what they are doing. Was it an in-house solution, something from a startup, or something from an established enterprise tech company? One of the things I’ve been saying repeatedly is that companies are going to implement machine learning incorrectly, and there will be obvious failures and blow ups, but the most dangerous and catastrophic failures will be a cascade of things that look like they work correctly. In this case, Tumblr implemented something that was blatantly obviously broken. That’s just pathetically poor management. In the next 10 years, I would give high odds of least 1 company in the s&p 500 will fail due to bad machine learning implementation(s.)
- lmkg 8y ago
  > from an established enterprise tech company My first guess, based on basically nothing at all, was that this was an IBM Watson project. 0.3 seconds of Googling found that IBM has already published a case study on their website about Verizon using Watson. So, I'm going to go with that. https://www.ibm.com/case-studies/verizon https://www.ibm.com/case-studies/verizon
- pram 8y ago
  They previously implemented a flag feature that users could use to mark NSFW content. That was in 2017. I doubt there’s any ML, it’s probably just people flagging everything for the fun of it.

by dpwm 8y ago

newen 8y ago

Oh god...I remember my first project I just ran an ML algorithm against the dataset, got high accuracy, boasted to colleagues, and then explored the dataset only to realize it was highly unbalanced. So embarrassing.

AJ007 8y ago

I would really, really like to know what they are doing. Was it an in-house solution, something from a startup, or something from an established enterprise tech company? One of the things I’ve been saying repeatedly is that companies are going to implement machine learning incorrectly, and there will be obvious failures and blow ups, but the most dangerous and catastrophic failures will be a cascade of things that look like they work correctly. In this case, Tumblr implemented something that was blatantly obviously broken. That’s just pathetically poor management. In the next 10 years, I would give high odds of least 1 company in the s&p 500 will fail due to bad machine learning implementation(s.)

lmkg 8y ago
> from an established enterprise tech company My first guess, based on basically nothing at all, was that this was an IBM Watson project. 0.3 seconds of Googling found that IBM has already published a case study on their website about Verizon using Watson. So, I'm going to go with that. https://www.ibm.com/case-studies/verizon https://www.ibm.com/case-studies/verizon
pram 8y ago
They previously implemented a flag feature that users could use to mark NSFW content. That was in 2017. I doubt there’s any ML, it’s probably just people flagging everything for the fun of it.