Agree with sibling commenters that editors shouldn't be doing this themselves -- in my field, for instance, an editor is just the paid-in-prestige prof who agre

Agree with sibling commenters that editors shouldn't be doing this themselves -- in my field, for instance, an editor is just the paid-in-prestige prof who agrees to look through articles submitted in a certain area, find referees for the paper, cajole the referees to review it at 6 months, cajole the referees again at one year, etc., and then make a decision based on the referee reports. (Clearly I'm not in CS as a 1.5-year turnaround is not so nuts in my field.) Anyhow, I think Qworg has a point in that the editors do have responsibility for what they accept. It's true that referees do not have the resources to reproduce results: I for instance have limited time on the shared high-performance computing resources we have, so I'm not going to use it on someone else's paper. However, as another commenter says, this could be paid for. A journal could potentially "sell" this as a competitive advantage: papers in this journal were actually reproduced, by paid staff who are neither editors nor referees, people paid in currency that can be exchanged for goods (not academic currency).