2 ms·
There are different types of research. Negative AI results are not like Michelson-Morley experiment in physics that can positively establish the lack of somethi
by bonoboTP 3mo ago
There are different types of research. Negative AI results are not like Michelson-Morley experiment in physics that can positively establish the lack of something. It's more of a craft, the "make it work" style of thing. Give the same task or goal to two different teams, you will see massively different outcomes. Simply failing to make it work is not conclusive.
To make such a negative result acceptable in AI, you'd have to have some clear reason why you think that your particular setup should produce the result you want, that exact configuration and architecture, dataset etc. There are countless projects in AI that fail. And it's not clear at all that it refutes any abstract hypothesis. It's a get-your-hands-dirty field. It can make or break a project whether someone has that tacit knowledge, that black magic experience to know how to properly do the project.
People can generate extremely many ideas. You'd need to convince me that your idea (among a million others that people are trying each day) is so significant that its failure is in itself interesting. If you were to review for AI conferences, you'd see the flood of papers that claim to achieve 0.5% or 1% improvement on some benchmark. Now imagine that they didn't even have that to show for it. It got worse by 2% after trying their random idea. Who cares then? Even the +1% with a random idea is quite annoying to accept. But if their random idea really made something work much better, I will at least have some reason to want to see what may be going on there, there can be some signal. With negative results, it's very uninteresting.
- don_esteban 3mo agoI did read OP's post much more generally than only about AI research. There are many research areas where (properly done!) negative results would be very valuable, but are not properly encouraged. I do agree that in a new area with too many degrees of freedom (and yes, AI research is one of those), negative results (especially poorly done) are of limited usefulness.