4 ms·
The test labels are not known to them. The test labels are held by a different team that tells them their test error rate. Furthermore, multiple, separate teams
by argonaut 11y ago
The test labels are not known to them. The test labels are held by a different team that tells them their test error rate. Furthermore, multiple, separate teams have surpassed human performance.
FWIW, "human performance" is not as meaningful as most people make it out to be, because this is such a narrow task.
- tim333 11y agoThe paper mentions it's unfair to humans to have to tell say a coucal from an indigo bunting (both blue coloured birds). I've never heard of either so I guess I would have got that wrong but it's not the fault of my visual system.
- p1esk 11y agoBut you only have to guess right once in 5 tries!
- natch 11y agoI replied to abrichr's link. Did you look at it at all? They used a dataset from 2012. Are you saying that the test labels from a 2012 dataset have been kept secret until now? Really?
- argonaut 11y agoI am familiar with the ImageNet competition. Let me know if you find the ImageNet test labels. (Hint: they are not released, since they are re-used for competitions).
- natch 11y agoInteresting, did not know that. Do you think that proves this team was not able to get and exploit some inside knowledge? Judging from recent competitions, ethics does not seem to be a strong point in some cultures.