6 ms·

If there are bugs in your code that would have prevented you from being able to do what you claim to do, then to the readers of your paper, it seems more likely

If there are bugs in your code that would have prevented you from being able to do what you claim to do, then to the readers of your paper, it seems more likely that you might not have performed the experimentation that you claim to.

draw_down 7y ago
I'd say that depends on the readers an awful lot.
throwaway287391 7y ago
What? This has nothing to do with bugs in the code that would've affected the original author's ability to run the original experiments described in the paper. The user I replied to is talking about things like hard-coded data paths and wanting to be able to run the code at the press of a button on any platform. The expectation for this level of cross-platform compatibility is difficult to achieve for engineering firms with millions to invest in making it work properly. He's mad that individual machine learning researchers don't always do a great job of making it happen for their cutting-edge research which was a difficult enough engineering task to get running on their machine.
- gcommer 7y ago
  Reread the original post: > I have to fix bugs before the code can run. I'm very curious how did the author run that code with the bugs.
  - throwaway287391 7y ago
    You're right, I misread that part.
    - jfoutz 7y ago
      throwaway287391 it's hilarious how much you value your own opinion. You feel it's important your interlocutor knows you know you were corrected, while remaining anonymous. This is like, the funniest thing i've seen all week. You should run for office. _edit_ Mmmm delicious downvotes. How many alts do you have throwaway287391? i don't think dang would or should expose you. But it is even more funny to me that you would log in with an alt (2?!), specifically to downvote. Anyway.I'm sure with some practice you'll get better at trolling little buddy.
      - throwaway287391 7y ago
        I don't have much to say to most of this, but I'll point out that I didn't and couldn't have downvoted you as none of my throwaways have >=500 karma or whatever the threshold is.
        jfoutz 7y ago
        > I'll point out that I didn't and couldn't have downvoted you as none of my throwaways have >=500 karma I'll take you at your word. You want to just drop in and drop bombs and pretend that the information you are presenting is in some sense useful. Other folks pointed out that you are wrong. But you also want the reputation. You want the respect for revising your opinion. You're you from your perspective, but from everyone else's perspective you're not you, you're the basest troll that has ever posted on HN, because you have no rep, no history, not provenance. It's funny to me, because you want it both ways. It's pretty hilarious. I shouldn't feed the trolls. I know better. Enjoy being a complete waste of time to everyone that glances at your writing!
        throwawayjava 7y ago
        Man, I hate when I get dragged into these sorts of intense emotional downs by silly internet conversations with strangers. FWIW I undowned your posts; internet karma is about communication/moderation, and sometimes has the opposite of the intended effect. I've been there. Anyways, find some time to pet a dog/cat or fly a kite today.
      - romwell 7y ago
        FIY: elsewhere on Earth, admitting to being wrong is considered to be a commendable thing. Welcome to the planet, enjoy your stay!
      - jonas21 7y ago
        FWIW, I downvoted because because you somehow managed to break about half of the HN comment guidelines [1] in a single comment, in particular: * Be kind. Don't be snarky. Comments should get more thoughtful and substantive, not less, as a topic gets more divisive. * When disagreeing, please reply to the argument instead of calling names. "That is idiotic; 1 + 1 is 2, not 3" can be shortened to "1 + 1 is 2, not 3." * Please respond to the strongest plausible interpretation of what someone says, not a weaker one that's easier to criticize. Assume good faith. * Please don't make insinuations about astroturfing. It degrades discussion and is usually mistaken. If you're worried, email us and we'll look at the data. * Please don't comment about the voting on comments. It never does any good, and it makes boring reading. [1] https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
randcraw 7y ago
Agreed. If your results are not reproducible by others, the publication should issue a retraction and withdraw your work. Code that works is essential.
- Volt 7y ago
  I hope you're a scientist.
- dodobirdlord 7y ago
  Nonsense! The "reproducible" in "reproducible result" doesn't mean anything except that what is described in the paper can be reproduced. Code that works is nice and all, but totally irrelevant to the question of reproducibility. Would you claim that that the Higgs paper must be retracted because CERN won't ship you the LHC? They've got ATLAS and CMS to demonstrate reproducibility, and they're wildly different!
p1esk 7y ago
Typically when I post code for my papers, I try to clean it up, because it's an ugly mess that is hard to understand (even myself sometimes). This clean up process might introduce bugs. I'm not going to thoroughly test everything, I have next paper deadline coming up and my adviser could not care less about my code. But, if you're interested in my research and having trouble reproducing my results, email me and I'll help you.
- throwawayjava 7y ago
  ...and university/company sometimes makes you wipe out the git commit log, so you can't always just use "HEAD at time of submission". Pro tip: at paper submission time, md5sum your code and also git tag it in your private repo. When you release the code, if you have to / want to release with a clean history, make submission-time your initial commit and then make the current state of the repo your second commit. I've never encountered an institution that won't allow that level of history in the code release, even places that are pretty hard core about wiping pre-release history.
- caenorst 7y ago
  And don't you think it would have been more efficient to write it down clean at first with good comments and code practice ? Those good practices are here for a reason and not just for annoying you for the sack of "beauty". You would save more time for yourself when you wanna expand the paper further or just reuse some part of it, have better reputation for your work, and gain some citation because people will have the tendency to use your code as a starting point.
  - inimino 7y ago
    "Writing it down clean at first" implies that you know what is going to work when you start. If that's the case, it's not research.
  - patrick5415 7y ago
    No, it would not be more efficient. I’ve tried this route and it hands down consumes way more time. Until it’s clear that something will get re-used, it’s most efficient to leave it dirty. The software types love to sht on researchers for their poor quality code. Yet a few threads over, there’s always a discussion about how more tests don’t make it onto the kanban board (or whatever you guys are using these days) when crunch time approaches. It’s not just us.

draw_down 7y ago

I'd say that depends on the readers an awful lot.

throwaway287391 7y ago

What? This has nothing to do with bugs in the code that would've affected the original author's ability to run the original experiments described in the paper. The user I replied to is talking about things like hard-coded data paths and wanting to be able to run the code at the press of a button on any platform. The expectation for this level of cross-platform compatibility is difficult to achieve for engineering firms with millions to invest in making it work properly. He's mad that individual machine learning researchers don't always do a great job of making it happen for their cutting-edge research which was a difficult enough engineering task to get running on their machine.

gcommer 7y ago
Reread the original post: > I have to fix bugs before the code can run. I'm very curious how did the author run that code with the bugs.
- throwaway287391 7y ago
  You're right, I misread that part.
  - jfoutz 7y ago
    throwaway287391 it's hilarious how much you value your own opinion. You feel it's important your interlocutor knows you know you were corrected, while remaining anonymous. This is like, the funniest thing i've seen all week. You should run for office. _edit_ Mmmm delicious downvotes. How many alts do you have throwaway287391? i don't think dang would or should expose you. But it is even more funny to me that you would log in with an alt (2?!), specifically to downvote. Anyway.I'm sure with some practice you'll get better at trolling little buddy.
    - throwaway287391 7y ago
      I don't have much to say to most of this, but I'll point out that I didn't and couldn't have downvoted you as none of my throwaways have >=500 karma or whatever the threshold is.
      - jfoutz 7y ago
        > I'll point out that I didn't and couldn't have downvoted you as none of my throwaways have >=500 karma I'll take you at your word. You want to just drop in and drop bombs and pretend that the information you are presenting is in some sense useful. Other folks pointed out that you are wrong. But you also want the reputation. You want the respect for revising your opinion. You're you from your perspective, but from everyone else's perspective you're not you, you're the basest troll that has ever posted on HN, because you have no rep, no history, not provenance. It's funny to me, because you want it both ways. It's pretty hilarious. I shouldn't feed the trolls. I know better. Enjoy being a complete waste of time to everyone that glances at your writing!
        throwawayjava 7y ago
        Man, I hate when I get dragged into these sorts of intense emotional downs by silly internet conversations with strangers. FWIW I undowned your posts; internet karma is about communication/moderation, and sometimes has the opposite of the intended effect. I've been there. Anyways, find some time to pet a dog/cat or fly a kite today.
    - romwell 7y ago
      FIY: elsewhere on Earth, admitting to being wrong is considered to be a commendable thing. Welcome to the planet, enjoy your stay!
    - jonas21 7y ago
      FWIW, I downvoted because because you somehow managed to break about half of the HN comment guidelines [1] in a single comment, in particular: * Be kind. Don't be snarky. Comments should get more thoughtful and substantive, not less, as a topic gets more divisive. * When disagreeing, please reply to the argument instead of calling names. "That is idiotic; 1 + 1 is 2, not 3" can be shortened to "1 + 1 is 2, not 3." * Please respond to the strongest plausible interpretation of what someone says, not a weaker one that's easier to criticize. Assume good faith. * Please don't make insinuations about astroturfing. It degrades discussion and is usually mistaken. If you're worried, email us and we'll look at the data. * Please don't comment about the voting on comments. It never does any good, and it makes boring reading. [1] https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html

randcraw 7y ago

Agreed. If your results are not reproducible by others, the publication should issue a retraction and withdraw your work. Code that works is essential.

Volt 7y ago
I hope you're a scientist.
dodobirdlord 7y ago
Nonsense! The "reproducible" in "reproducible result" doesn't mean anything except that what is described in the paper can be reproduced. Code that works is nice and all, but totally irrelevant to the question of reproducibility. Would you claim that that the Higgs paper must be retracted because CERN won't ship you the LHC? They've got ATLAS and CMS to demonstrate reproducibility, and they're wildly different!

p1esk 7y ago

Typically when I post code for my papers, I try to clean it up, because it's an ugly mess that is hard to understand (even myself sometimes). This clean up process might introduce bugs. I'm not going to thoroughly test everything, I have next paper deadline coming up and my adviser could not care less about my code. But, if you're interested in my research and having trouble reproducing my results, email me and I'll help you.

throwawayjava 7y ago
...and university/company sometimes makes you wipe out the git commit log, so you can't always just use "HEAD at time of submission". Pro tip: at paper submission time, md5sum your code and also git tag it in your private repo. When you release the code, if you have to / want to release with a clean history, make submission-time your initial commit and then make the current state of the repo your second commit. I've never encountered an institution that won't allow that level of history in the code release, even places that are pretty hard core about wiping pre-release history.
caenorst 7y ago
And don't you think it would have been more efficient to write it down clean at first with good comments and code practice ? Those good practices are here for a reason and not just for annoying you for the sack of "beauty". You would save more time for yourself when you wanna expand the paper further or just reuse some part of it, have better reputation for your work, and gain some citation because people will have the tendency to use your code as a starting point.
- inimino 7y ago
  "Writing it down clean at first" implies that you know what is going to work when you start. If that's the case, it's not research.
- patrick5415 7y ago
  No, it would not be more efficient. I’ve tried this route and it hands down consumes way more time. Until it’s clear that something will get re-used, it’s most efficient to leave it dirty. The software types love to sht on researchers for their poor quality code. Yet a few threads over, there’s always a discussion about how more tests don’t make it onto the kanban board (or whatever you guys are using these days) when crunch time approaches. It’s not just us.