4 ms·

"Winning" by anything that can reasonably called cheating, as in this case, does not advance the general state of the art. Innovation is best served through app

"Winning" by anything that can reasonably called cheating, as in this case, does not advance the general state of the art. Innovation is best served through appropriate rules and competition structure.

sytelus 7y ago
Yes, and that's the right thing to do in the academic research setting ("advance the state of the art"). But the public competitions with monetary rewards are not the same setting. I can imagine scenarios where the guy stole the test set from Kaggle servers (i.e. unlawful access) should disqualify him permanently. But the essence of the competition should be the focus on cracking a given problem, not about a specific technique. One test a good of ML competition: Can it be solved by simply hiring lots of humans to make predictions without incurring significantly more costs than the prize money?
- deleted 7y ago
  [deleted]
- joe_the_user 7y ago
  One test a good of ML competition: Can it be solved by simply hiring lots of humans to make predictions without incurring significantly more costs than the prize money? What value to the organizers, to society or to whatever are you imagining coming out of a free-for-all style competition? I think the organizers now imagine that the result would identifying good, generic prediction algorithms along with identifying good AI programmers capable of producing general prediction algorithms. It seems like the contest framework already has become a bit problematic through context winners just being good at contests and not otherwise achieving anything. But what are you thinking of? There are already hacking competitions btw.
sytelus 7y ago
Again, I'm not defending Pleskov. If he had come forth with the hack, things would have been different. Instead, he pretended that he had ML solution, pocket the money and put an extraordinary effort into making sure that people can't actually figure out his true doings. He was disingenuous, fully self-aware that he was in the wrong and did his best to cover up his tracks. It wasn't fair to other competitor and it was most certainly not fair to the organization trying to do something good. So yes, Pleskov, remains indefensible.
sdenton4 7y ago
I disagree... The hope with competitions is that we learn something new through massive semi collaborative exploration. Raising the barrier to hosting useful competitions means we learn less.
sjg007 7y ago
You see the same thing in grad school.
- mannykannot 7y ago
  Indeed you do, and I know someone who got screwed by a couple of plagiarizing cheaters. This probably contributed to his abandoning his studies and possibly to his suicide. I take a dim view of suggestions that cheating is reasonable, let alone that it is the smart option.

sytelus 7y ago

Yes, and that's the right thing to do in the academic research setting ("advance the state of the art"). But the public competitions with monetary rewards are not the same setting. I can imagine scenarios where the guy stole the test set from Kaggle servers (i.e. unlawful access) should disqualify him permanently. But the essence of the competition should be the focus on cracking a given problem, not about a specific technique. One test a good of ML competition: Can it be solved by simply hiring lots of humans to make predictions without incurring significantly more costs than the prize money?

deleted 7y ago
[deleted]
joe_the_user 7y ago
One test a good of ML competition: Can it be solved by simply hiring lots of humans to make predictions without incurring significantly more costs than the prize money? What value to the organizers, to society or to whatever are you imagining coming out of a free-for-all style competition? I think the organizers now imagine that the result would identifying good, generic prediction algorithms along with identifying good AI programmers capable of producing general prediction algorithms. It seems like the contest framework already has become a bit problematic through context winners just being good at contests and not otherwise achieving anything. But what are you thinking of? There are already hacking competitions btw.

sytelus 7y ago

Again, I'm not defending Pleskov. If he had come forth with the hack, things would have been different. Instead, he pretended that he had ML solution, pocket the money and put an extraordinary effort into making sure that people can't actually figure out his true doings. He was disingenuous, fully self-aware that he was in the wrong and did his best to cover up his tracks. It wasn't fair to other competitor and it was most certainly not fair to the organization trying to do something good. So yes, Pleskov, remains indefensible.

sdenton4 7y ago

I disagree... The hope with competitions is that we learn something new through massive semi collaborative exploration. Raising the barrier to hosting useful competitions means we learn less.

sjg007 7y ago

You see the same thing in grad school.

mannykannot 7y ago
Indeed you do, and I know someone who got screwed by a couple of plagiarizing cheaters. This probably contributed to his abandoning his studies and possibly to his suicide. I take a dim view of suggestions that cheating is reasonable, let alone that it is the smart option.