4 ms·
Well there are entire areas of CS research tangential to PLs (say, SAT/SMT, software verification, program synthesis) where the fundamental problems are known t
by lou1306 3y ago
Well there are entire areas of CS research tangential to PLs (say, SAT/SMT, software verification, program synthesis) where the fundamental problems are known to be NP-complete/exponential/undecidable. So it is pretty hard to declare a new approach "superior" on purely theoretical grounds: you usually have to
run some benchmarks and see how your new approach compares to existing alternatives. And you want these benchmark to be replicable across different machines and platforms.
- mike_hearn 3y agoYes performance involves experiments, but are they scientific experiments? I'm not sure it matters either way, it's a purely semantic debate. The issues that create replicability problems in CS are pretty different to the ones that create issues in other fields. My experience was that they're purely engineering problems rather than problems of incentives or non-replicable designs. If the issue was just that some papers occasionally don't replicate because the authors forgot a detail, they get queried and update the paper then nobody would care about this issue. It gets attention because that's sadly not what happens.