3 ms·
I'm a scientist from a field outside ML who knows that ML can contribute to science. But I'm also really sad to see false claims in papers. For example, a goo
by dekhn 8y ago
I'm a scientist from a field outside ML who knows that ML can contribute to science. But I'm also really sad to see false claims in papers. For example, a good scientist can read an ML paper, see claims of 99% accuracy, and then probe further to figure out what the claims really mean. I do that a lot, and I find that accuracy inflation and careless mismanagement of data mars most "sexy" ML papers. To me, that's what's going to lead to a new AI winter.
- fmap 8y agoI'm in the same situation and it's really worrying. Deep learning is the method of choice for a number of concrete problems in vision, nlp, and some related disciplines. This is a great success story and worthy of attention. Another AI winter will just make it harder to secure funding for something that may well be a good solution to some problems.
- mtgx 8y agoYou hear Facebook all the time saying how it "automatically blocks 99% of the terrorist content" with AI to the public and governments. Nobody thought to ask: "How do you know all of that content is terrorist content? Does anyone check every video afterwards to ensure that all the blocked content was indeed terrorist content?" (assuming they even have an exact definition for it).
- shmageggy 8y agoAlso, how do they know how much terrorist content they aren't blocking (the 1%), since they by definition haven't found it yet?
- jaggednad 8y agoYes, they have a test set. They don’t check every video, but they do check every video in a sample that is of statistically significant size.
- varelse 8y agoI'm 100% convinced it can block 99% of all terrorist content that hasn't been effectively SEOed to get around their filters because that's just memorizing attributes of the training set data. Unfortunately, the world isn't a stationary system like these ML models (usually) require. I still get spam in my gmail account, nowhere near as much as I do elsewhere, but I still get it.
- fantasticsid 8y ago> Does anyone check every video afterwards to ensure that all the blocked content was indeed terrorist content? They might not, but they could sample them to be statistically confident?