6 ms·
The title is linkbait: There is no cartel. The word cartel implies collusion. There is no collusion. Using the word "cartel" in this case is not only wrong, b
by cs702 5y ago
The title is linkbait: There is no cartel.
The word cartel implies collusion. There is no collusion.
Using the word "cartel" in this case is not only wrong, but also... insulting to all the hardworking individuals who have toiled away in obscurity to compile, clean-up, label, package, and publish influential datasets.
Despite the terrible title, the OP is worth a read. It summarizes a recent research paper, also worth a read, about the disproportionate popularity in ML research of a relatively small number of datasets sponsored by and produced at elite institutions (e.g., ImageNet for visual recognition):
https://openreview.net/forum?id=zNQBIBKJRkd https://openreview.net/forum?id=zNQBIBKJRkd
Surely we can find ways to address the winner-take-all dynamics of benchmark popularity without unfairly accusing anyone of running a cartel.
- throwawaygh 5y ago> Surely we can find ways to address the winner-take-all dynamics of benchmark popularity Reviewer 2 is probably the only/best way to address this. More-so if they're a grant reviewer, of course, but paper reviewers will do ok as well.