2 ms·
It does, but I think the past performance is probably legitimate: the hopscotch nature of the Imagenet gains (in which it seemed like every month a different de
by gwern 11y ago
It does, but I think the past performance is probably legitimate: the hopscotch nature of the Imagenet gains (in which it seemed like every month a different deep-learning team would turn in a new record) means that we can be confident that the overall performance curve of all teams is genuine unless they are all colluding, and this particular method of cheating previously can be ruled out just by looking at the submission scatterplot and cumulative submissions in the official statement ( http://www.image-net.org/challenges/LSVRC/announcement-June-2-2015 http://www.image-net.org/challenges/LSVRC/announcement-June-... ) - we can be sure that competitors and previous records did not overfit like this because there just aren't a whole lot of previous submissions to the test set.
So now the record is probably more like 4.5% than the 4% or whatever Baidu's final paper was claiming. To keep this in perspective, that's a few months or a year of progress, the Imagenet record is still human-equivalent, and deep learning is amazing.