It's been a few years since this was relevant for me, but my primary concern would be that in order to publish actual code I would either have to spend months c

It's been a few years since this was relevant for me, but my primary concern would be that in order to publish actual code I would either have to spend months cleaning it up, writing documentation etc., or spend the next year fielding support calls. Or both. I agree with another poster here that having somebody else repeating the experiment with their own implementation is a better test for validity - if a second paper just copies the source code from the first and makes a few tweaks, mistakes could easily carry over. But having the data available would be great.