I'm not sure if I have exactly the same concerns as the person you're replying to, but I've definitely noticed problems coming from this form of "replication" t

I'm not sure if I have exactly the same concerns as the person you're replying to, but I've definitely noticed problems coming from this form of "replication" through just re-running existing code. I don't think that means code shouldn't be released, but I think we should be wary of scientific code being reused verbatim too frequently. If existing code is buggy or doesn't do what its authors think it does, this can be a big problem. Even if the idea is correct, whole swathes of downstream literature can get contaminated if authors didn't even attempt to implement an algorithm independently, but just blindly reused an existing docker image that they didn't understand. I consider that poor practice. If you claim that your work is based on Paper X, but what you really mean is that you just ran some Docker image associated with Paper X and trusted that it does what it says it does, instead of independently reimplementing the algorithm, that is not replication. In older areas of science the norm is to try to replicate a paper from the published literature without reuse of "apparatus". For example, in chemistry you would try to replicate a paper using your own lab's glassware, different suppliers for chemicals, different technicians, etc. Analogies here are imperfect, but I would consider downloading and re-running an existing docker image to be dissimilar from that. It's more like re-running the experiment in the first lab rather than replicating the published result in a new lab. As a result it can, like re-running a chemistry experiment in the same lab, miss many possible confounds. Of course, you do have to stop somewhere. Chemistry labs don't normally blow their own glass, and it's possible that the dominant way of making glass in your era turns out to be a confound (this kind of thing really does happen sometimes!). But imo, on the code side of things, "download and rerun a docker image" is too far towards the side of not even trying to replicate things independently.