3 ms·
Specifically to the CS context, I think some version of double blinded peer code-review should be made mandatory for a publication. I've seen authors skip quit
by binaryzeitgeist 10y ago
Specifically to the CS context, I think some version of double blinded peer code-review should be made mandatory for a publication.
I've seen authors skip quite a many details that are quintessential to the replication process.
In short, if research is not replicable by the peer community it just useless, that's what it is.
- jedberg 10y agoCS is the craziest of them all. Those should be the easiest to replicate. "Here is the code, here is a manifest of the environment/container/disk image/etc." You should be able to take that and run it and get the same results. Or are you saying that the code itself is the problem and that they've done the equivalent of "return True" to get the result they want?
- tnecniv 10y agoIn my other comment I mentioned the CS results I've largely struggled to reproduce is because they include enough detail for you to get the gist of how it works, but not enough to avoid going down some rabbit holes. Also, not all publications include code. Many venues don't require it.