3 ms·

I am glad someone is doing this. I recently tried to implement an algorithm described in a paper in a well known chemistry journal and discovered that the paper

I am glad someone is doing this. I recently tried to implement an algorithm described in a paper in a well known chemistry journal and discovered that the paper's results were incorrect. It turns out that the author's program (which was not made available with the paper) didn't correspond with what was described in the paper. That is to say, when I used the described algorithm, I got very different results than those in the paper. Now I'm not sure what to do. I want to correct the paper, but I don't want to openly criticize the work of others.

corford 14y ago
If you are prepared to back your claim up in a way that can be independently verified, why don't you want to criticize the original authors work? Surely correct science is more important than sparing someone's blushes?
- montecarl 14y ago
  After reading your comment, I can't come up with any reason other than the blushes. Perhaps I'll reconsider.
  - otoburb 14y ago
    How about reaching out to the original author and outlining the problems that you've been encountering? This way, you can at least give them a chance to respond privately or correct potential misunderstandings with the paper (perhaps publicly). Similar analogy would be approaching a vendor regarding a security/vulnerability disclosure.
    - montecarl 14y ago
      I already have. They recently gave me the source code and I pointed out the problems. They agree that there are bugs in their code. I just haven't approached them about publishing a comment on their paper because I feel bad about it.
nchlswu 14y ago
While this is part of the 'problem' that is being solved by this project, I think as a whole, this issue is separate from the intent of that project. Making experiments transparent (ie. including data sets, including the algorithm) is an extremely important issue and given the increased reliance on software in these experiments, it is important to make them available. I'd wager that in this project, many of the reproductions will be contingent on using the exact same apparatus. Unfortunately not enough information is usually made available to reproduce.

corford 14y ago

If you are prepared to back your claim up in a way that can be independently verified, why don't you want to criticize the original authors work? Surely correct science is more important than sparing someone's blushes?

montecarl 14y ago
After reading your comment, I can't come up with any reason other than the blushes. Perhaps I'll reconsider.
- otoburb 14y ago
  How about reaching out to the original author and outlining the problems that you've been encountering? This way, you can at least give them a chance to respond privately or correct potential misunderstandings with the paper (perhaps publicly). Similar analogy would be approaching a vendor regarding a security/vulnerability disclosure.
  - montecarl 14y ago
    I already have. They recently gave me the source code and I pointed out the problems. They agree that there are bugs in their code. I just haven't approached them about publishing a comment on their paper because I feel bad about it.

nchlswu 14y ago

While this is part of the 'problem' that is being solved by this project, I think as a whole, this issue is separate from the intent of that project. Making experiments transparent (ie. including data sets, including the algorithm) is an extremely important issue and given the increased reliance on software in these experiments, it is important to make them available. I'd wager that in this project, many of the reproductions will be contingent on using the exact same apparatus. Unfortunately not enough information is usually made available to reproduce.