3 ms·
> Another kind of fake result I've encountered which is even less directly a result of the software engineering shortage is in signal processing literature. Whi
by izacus 7y ago
> Another kind of fake result I've encountered which is even less directly a result of the software engineering shortage is in signal processing literature. While working on audio/video compression I found it common for algorithms to be presented without various constants and after reimplementing and asking the authors for their constants I found that they'd been cherrypicked for the ten images used in the paper, and that the whole approach doesn't actually work. This is a kind of ineptitude (or outright dishonesty) that would be much less common in a world where reviewers received a working and usable implementation in source form-- but that can't be expected in a world where qualified software engineering is not readily available to researchers.
This happened so many times when I was working on NLP/CV algorithms - I've read and implemented many algorithms from papers just to find out that they only produce the amazing improvements on cherrypicked dataset. On pretty much all other practical data the algorithms performed worse and in many cases even crashed!