3 ms·

(edit the "code" I'm talking about is complex stuff like numerical simulations etc, if we were talking about the code used for plotting a simple graph or histog

(edit the "code" I'm talking about is complex stuff like numerical simulations etc, if we were talking about the code used for plotting a simple graph or histogram, that's a different matter--although it wouldn't hurt anyone publishing a full paper open source as a make-file with some TeX sources calling a few plotting libraries) > But they are presenting the methodology they claim went into the code and allowing you to agree or disagree with the methodology. If you agree, you take their answer-- No. And this is the whole point. You can agree with the methodology, but as long as you haven't inspected the code, you cannot just accept they actually implemented it right! In fact, chances are, they probably didn't. The best programmers white buggy and incorrect code. And from what I've seen in the field of physics, scientific code is everything but an exception to that. I'd be surprised if it was very different for astrophysics. So by publishing the methodology and the results, but not the code, they put up a nice show. But that's it, not reproducible. So all that's really brought to the table is the methodology and some "say-so" results that nobody can check are accurate.

anon_barcode 14y ago
True. Nobody can check if your data + your methodology = the results you claim. But it is quite common for someone to take the same problem, collect their own data and run their own analysis (both steps can be completely different from the original statement-- collecting data of a different type, from a different instrument, etc. Running an analysis of a different paradigm, running the same analysis to a different precision, running the same analysis via a different algorithm, etc.). If you read the arxiv on a daily basis you can see huge academic arguments unfolding over the course of months and years. There seems to be this idea that the conversation goes: "Yo I found this hypervelocity star" "Dope, let's move on" It's actually more like: "Yo I found this hypervelocity star" "Nope, I got spectra and you're wrong" "Well I got ultra high-res spectra and I think he's right" "Actually all of you are forgetting asymmetric drift, this is just a geometry problem, l2angles" "Hey, I sit in my basement and play with MOND, it might help" "My simulations show something completely different" Authors are called out and proven right or wrong on a daily basis, even if we can't watch them code over the shoulder. I actually think that's one of the beauties of it-- most of our methods are invented by trying to prove or disprove something in a new way. The monster codes like GADGET (which ran the millenium sim everyone's seen) are usually made public after ~5 years of proprietariness.
- tripzilch 14y ago
  > It's actually more like: "Yo I found this hypervelocity star" "Nope, I got spectra and you're wrong" "Well I got ultra high-res spectra and I think he's right" "Actually all of you are forgetting asymmetric drift, this is just a geometry problem, l2angles" "Hey, I sit in my basement and play with MOND, it might help" "My simulations show something completely different" Haha, that actually sounds almost exactly like the Stone Soup parable :) ( http://en.wikipedia.org/wiki/Stone_soup http://en.wikipedia.org/wiki/Stone_soup )

anon_barcode 14y ago

True. Nobody can check if your data + your methodology = the results you claim. But it is quite common for someone to take the same problem, collect their own data and run their own analysis (both steps can be completely different from the original statement-- collecting data of a different type, from a different instrument, etc. Running an analysis of a different paradigm, running the same analysis to a different precision, running the same analysis via a different algorithm, etc.). If you read the arxiv on a daily basis you can see huge academic arguments unfolding over the course of months and years. There seems to be this idea that the conversation goes: "Yo I found this hypervelocity star" "Dope, let's move on" It's actually more like: "Yo I found this hypervelocity star" "Nope, I got spectra and you're wrong" "Well I got ultra high-res spectra and I think he's right" "Actually all of you are forgetting asymmetric drift, this is just a geometry problem, l2angles" "Hey, I sit in my basement and play with MOND, it might help" "My simulations show something completely different" Authors are called out and proven right or wrong on a daily basis, even if we can't watch them code over the shoulder. I actually think that's one of the beauties of it-- most of our methods are invented by trying to prove or disprove something in a new way. The monster codes like GADGET (which ran the millenium sim everyone's seen) are usually made public after ~5 years of proprietariness.

tripzilch 14y ago
> It's actually more like: "Yo I found this hypervelocity star" "Nope, I got spectra and you're wrong" "Well I got ultra high-res spectra and I think he's right" "Actually all of you are forgetting asymmetric drift, this is just a geometry problem, l2angles" "Hey, I sit in my basement and play with MOND, it might help" "My simulations show something completely different" Haha, that actually sounds almost exactly like the Stone Soup parable :) ( http://en.wikipedia.org/wiki/Stone_soup http://en.wikipedia.org/wiki/Stone_soup )