7 ms·
Microsoft Research wins image recognition competition
- dplarson 11y agoIt's mentioned in the article, but here's a direct link to the paper on arXiv: http://arxiv.org/abs/1512.03385 http://arxiv.org/abs/1512.03385 And direct link to the PDF: http://arxiv.org/pdf/1512.03385v1.pdf http://arxiv.org/pdf/1512.03385v1.pdf
- wyc 11y ago"Baidu does not show up in this year’s rankings. The company made more submissions than were permitted and ultimately apologized and fired the team leader who directed juniors to make the unacceptable submissions."
- escap 11y agosee http://dswalter.github.io/blog/machine-learnings-first-cheating-scandal/ http://dswalter.github.io/blog/machine-learnings-first-cheat... for some background on this.
- sherjilozair 11y agoMight be worth noting that it is very likely that Google, Facebook and Baidu did not even participate in this ImageNet. It is widely believed in the vision community that this dataset is at the end of its run, i.e. any subsequent improvements in performance are not due to scientific breakthroughs but through hyperparameter optimization and ensembling.
- Estragon 11y agoSurely that's not true until someone achieves human-level performance with their entry.
- abrichr 11y agoHuman level performance was surpassed earlier this year: http://arxiv.org/abs/1502.01852 http://arxiv.org/abs/1502.01852
- natch 11y agoI won't be surprised when human level performance is surpassed, but isn't that paper based on work that has the advantage of knowing the full test dataset in advance? You can say "yes but they only trained on the training data" but that doesn't rule out tweaking across several experiments and measuring them against the known test data, then cherry picking the experiment that was best overfit to that data, right? I'm not saying there are shenanigans here, just wondering how you know there are not.
- argonaut 11y agoThe test labels are not known to them. The test labels are held by a different team that tells them their test error rate. Furthermore, multiple, separate teams have surpassed human performance. FWIW, "human performance" is not as meaningful as most people make it out to be, because this is such a narrow task.
- 11y ago
- deleted 11y ago[deleted]
- mikeskim 11y agoIs there a private one shot hold out in this particular competition?
- smhx 11y agoyes. more like 5-shot hold-out, and the held-out set is pretty big as well, so more legit.
- mikeskim 11y agoso you mean there are 5 entries you can select to be scored in the end, and there is one private leaderboard no one can see but the admins? do you know the size of the private leaderboard test set?
- kadvani1 11y agoWe used this technology last week to build a Computer Vision TV https://news.ycombinator.com/item?id=10728973 https://news.ycombinator.com/item?id=10728973
- robgibbons 11y agoI'd like to have seen how IBM would have fared with Watson.
- tmerr 11y agoIsn't Watson a question answering system? I don't think it's specialized for image recognition.
- hiddencost 11y agohttps://www.ibm.com/smarterplanet/us/en/ibmwatson/developercloud/visual-recognition.html https://www.ibm.com/smarterplanet/us/en/ibmwatson/developerc... Watson is a marketing term. IBM provides a variety of machine learning services and calls them all Watson.
- tmerr 11y agoWell, that's just confusing.
- skadamat 11y agoWatson doesn't have much unique computer vision capabilities unfortunately. They did some good work in NLP a decade ago but not added much since
- douche 11y agoHave they not really added much to it since the Jeopardy stunt? We get some periodic rumblings about integrating it with a customer-service tool, but it is starting to sound a little Duke Nukem Forever
- vcarl 11y agoThey have it doing medical analysis (I'm hesitant to say "diagnosis" but I think it's being used to find symptom patterns doctors might otherwise miss). http://www.ibm.com/smarterplanet/us/en/ibmwatson/health/ http://www.ibm.com/smarterplanet/us/en/ibmwatson/health/ Watson for Oncology analyzes a patient’s medical information against a vast array of data, including ongoing expert training from MSK physicians, cancer case histories, and more, to provide evidence-based treatment options.
- geomark 11y agoI'm disappointed to hear this after watching the video of Andrew Ng's talk. He was very proud of Baidu's accomplishment in this competition.
- free2rhyme214 11y agoWhat products does Microsoft use this technology in? I've seen what Google's doing in Google Photos but I've noticed it's not accurate all the time.
- eru 11y agoSometimes even with political repercussions.
- jacquesm 11y ago> What products does Microsoft use this technology in? http://venturebeat.com/2015/11/11/microsoft-launches-project-oxford-apis-for-face-tracking-emotion-speaker-recognition-spell-checking/ http://venturebeat.com/2015/11/11/microsoft-launches-project...
- free2rhyme214 11y agoDeveloper API's. It would be nice to see a consumer use case in Windows.
- cmarschner 11y agoCan't speak for this particular team, but a submission for a competition is usually done on the last minute, so it's highly unlikely that _this_ algorithm is part of a product like Oxford API already. But a predecessor might, and it might get integrated over time. A research prototype usually requires a lot of duct tape to work. Productization usually requires a different mindset (i.e. an engineering rather than a research team) who care more about things like test coverage, distribution, (time) performance etc. So by definition, MSR is ahead of the curve, product teams are on the curve.
- Artemis2 11y agoIt could be used in Project Adam: https://youtu.be/zOPIvC0MlA4 https://youtu.be/zOPIvC0MlA4
- Artemis2 11y agoIt could be used in Project Adam: https://youtube.com/zOPIvC0MlA4 https://youtube.com/zOPIvC0MlA4
- dharma1 11y agoHas anyone implemented this yet for Tensorflow or Caffe? Also, any more details on the MSCOCO segmentation part?