4 ms·
I think it would be good to have some publishers require reproducibility, and some publishers that don't. Every requirement is exclusionary, and there are alway
by spencerchubb 2y ago
I think it would be good to have some publishers require reproducibility, and some publishers that don't. Every requirement is exclusionary, and there are always edge cases where a paper can be useful even if it doesn't fulfill a requirement.
- josephg 2y agoThat’s already the case. Eurosys includes an Artifact Evaluation for submitted papers - which typically includes code and data to allow reviewers to reproduce the work described in the paper. It’s optional, but encouraged. This page lists the artifact evaluation criteria for a handful of conferences: https://sysartifacts.github.io/ https://sysartifacts.github.io/
- ashton314 2y agoAs an intermediate point in the Require Reproducibility ↔ No Requirement spectrum, strong encouragement to have reproducible artifacts is attractive as well. I just got a paper accepted to ECOOP (European Conference on OO Programming; has lost it's OO-focus and is now a general PL conference) and it's trying something relatively new: artifact evaluations are considered as part of the submission process. Our paper had a reproducible and reusable artifact, and I think that helped our case with the reviewers.