3 ms·

Deeply disturbing fact. The main researcher in our group works in cutting edge numeric simulations, and although he does not publish his code (mostly because "y

Deeply disturbing fact. The main researcher in our group works in cutting edge numeric simulations, and although he does not publish his code (mostly because "you can code it yourself! It is really easy, listen" and go on for 3 hours), if you were to ask, he would give it to you without any problems. But according to what you say, this means that he (still programs in Fortran) would need to start caring about versioning, cross-platform stuff (fortran, remember), readability (well, it's fortran, for me it is always unreadable) and maintainability. In addition to his usual duties of "doing research", giving lectures, answering students, advising PhD's and asking for grants. The problem is most likely to be solved when everyone in the field can code the algorithm in the paper. If the peer reviewer can write a program following the algorithm and applied on the data it validates the results, this ends up working well. I doubt (very profoundly) that giving the source will result in better peer reviews. The peer reviewer may have the code, but he will most likely compile, run and check that the numbers are the same.

Maro 16y ago
In my group, we had two guys (me and another) who had a strong CS background, so we could advise on such issues. So, in general, the professor's strategy could be to simply make sure he always has such a person (Phd or postdoc) as part of his team, who advises on coding standards, versioning, cross platform issues, etc.
- RBerenguel 16y ago
  But then money is a huge problem (at least here in Spain). At most, you can have a mathematician who knows about this and can help, and it is pretty hard (I could on a basic level, but I know only two other people in our department using version control, for example).
Maro 16y ago
I doubt (very profoundly) that giving the source will result in better peer reviews. I strongly agree with you on this point. The peer reviewer obviously won't have the time to check the code. But the very act of releasing the code will force the original author to be more thorough. In this sense, it will result in better, less buggy code, better results and better science. Also, going one step beyond the peer reviewer, I think an interested party has the right to check the results for software and data bugs, given that we're in agreement that the peer reviewer probably won't have the time? Another anecdote. We found a really interesting signal in a popular astro dataset, which would have put us on the frontpage of a few magazines for sure. After many sleepless nights, it turned out that it was a bug in the processing pipeline software. It took me an unreasonable amount of time to find the bug, because the software wasn't released, so I had to go hunting around for clues in the appropriate papers. (This was a very large survey, so there were tons of papers.) If we had higher software engineering standards, maybe this bug would not have made it into the production pipeline and pollute the dataset.

Maro 16y ago

In my group, we had two guys (me and another) who had a strong CS background, so we could advise on such issues. So, in general, the professor's strategy could be to simply make sure he always has such a person (Phd or postdoc) as part of his team, who advises on coding standards, versioning, cross platform issues, etc.

RBerenguel 16y ago
But then money is a huge problem (at least here in Spain). At most, you can have a mathematician who knows about this and can help, and it is pretty hard (I could on a basic level, but I know only two other people in our department using version control, for example).

Maro 16y ago

I doubt (very profoundly) that giving the source will result in better peer reviews. I strongly agree with you on this point. The peer reviewer obviously won't have the time to check the code. But the very act of releasing the code will force the original author to be more thorough. In this sense, it will result in better, less buggy code, better results and better science. Also, going one step beyond the peer reviewer, I think an interested party has the right to check the results for software and data bugs, given that we're in agreement that the peer reviewer probably won't have the time? Another anecdote. We found a really interesting signal in a popular astro dataset, which would have put us on the frontpage of a few magazines for sure. After many sleepless nights, it turned out that it was a bug in the processing pipeline software. It took me an unreasonable amount of time to find the bug, because the software wasn't released, so I had to go hunting around for clues in the appropriate papers. (This was a very large survey, so there were tons of papers.) If we had higher software engineering standards, maybe this bug would not have made it into the production pipeline and pollute the dataset.