3 ms·

Couldn't the LSVRC just limit the amount of submissions? Why would you rely on the competence or good intentions for this?

• argonaut 11y ago
  The research community is a small one. Everyone knows everyone in the community.
• masklinn 11y ago
  > Couldn't the LSVRC just limit the amount of submissions? One of the article's notes indicates that it most likely does, and that the Baidu team wilfully got around that limit: > Members of the Baidu team had to create multiple logins in order to circumvent the “two submissions per week” rule
  • fsam 11y ago
    The fact that they created multiple logins was the biggest ethical breach here IMO. It was a willful violation of the contest rules. The absolute number of submissions is somewhat arbitrary, e.g. why are 40 okay, but 200 are not?
    • masklinn 11y ago
      There isn't an absolute limit on submissions, only a rate limit. The rate limit is probably somewhat arbitrary but it has a purpose: the aim of the challenge is to measure and compare algorithmic progress in machine learning, that's also why the test dataset is kept secret. Without that, it becomes possible to pre-tune the algorithm to more closely match or better recognise the specific dataset ("hyperparameter optimisation") but not work any better in the general case, so the submitter does better on the specific challenge, but doesn't actually advance the field. The limit is there to skew incentives towards algorithmic improvements, the specific rate doesn't really matter as long as it makes hyperparameter overfitting less convenient/efficient than algorithmic work.