4 ms·
What’s worse, sometimes the bugs are deliberate. A slight tuning of a parameter that can be explained away but having a significant impact on the output data. T
by Aperocky 7y ago
What’s worse, sometimes the bugs are deliberate. A slight tuning of a parameter that can be explained away but having a significant impact on the output data. This alone with statistical bullshiting ( there’s always a statistical manipulation that will create the desired result), renders 80% of all research practically worthless.
edit: code change -> parameter tuning.
- bonoboTP 7y agoOr in machine learning if that manipulation is not enough to get state of the art results, just don't cite anyone who is better! Now you're SOTA, congrats. Happens way more than people realize. Not as much in the top 2-3 conferences.
- GordonS 7y agoI can imagine this might happen in theory, but have you come across evidence of this in practice?
- Aperocky 7y agoParameter tuning is prevasive, but I guess I went too far to suggest that it's a code change.
- haihaibye 7y agoAnother common trick is to tune your program to a dataset then run all of the other programs with default parameters. Bam, you've got a benchmark where you're fastest, just like everyone else.