6 ms·

I live in this field now. I refuse to publish in so-called "highly-ranked journals" that take 3-6 months to find peers to review and referee your work, drag the

I live in this field now. I refuse to publish in so-called "highly-ranked journals" that take 3-6 months to find peers to review and referee your work, drag their feet, and basically amount to a ridiculous group of cronies that guard the gates of "science." I've talked about it before on HN but I will only publish to open access and open data (most important) journals that require all data be scrubbed of PII and published alongside the paper for replication purposes. I highly favor journals that not only allow, but encourage replication (yes, many elite journals cite "novelty" as a top factor regarding publication decisions... embarrassing). Science should be freely available, open access, open data, and replicable. Otherwise it isn't science. It's primarily garbage that exists to advance the careers and egos of a select few.

osrec 9y ago
Completely agree with you. I find it especially annoying when research funded by tax payers' money is not made freely available.
avip 9y ago
Don't forget code! "we then used our in-house untested code, developed by an undergrad we scrapped from Biology, to normalize the results. The code is comfortably not attached"
- kazinator 9y ago
  And that's just the CS papers.
  - ziotom78 9y ago
    Not entirely true: I am an astrophysicist, and I can say that this practice would be extremely useful for my discipline as well. In cosmology (my field) we use codes that are often extremely complex, yet they are not usually released together with the papers based on their outputs. This puzzles me, as last year I attended a conference on informatics and astrophysics (ADASS), and one of the talks showed that releasing your codes as open source increases a lot the chance of your papers being cited (expecially if you advertise your code using services like ASCL [1]). The only reason I can think of this unwillingness to publish codes is the fact that these codes written by physicists are often extremely unreadable: very long Fortran routines with awkward variable names, no comments, no test cases, no documentation… Once you get a result from your own software, you get more satisfaction if you publish a paper with the results than if you polish and publish the code. IMHO this is bad science, but it is difficult to change this way of working: cosmology is today done by large collaborations, not by individuals, and if you propose to make the codebase developed by your team public, this idea is usually not welcomed by your co-authors. [1] http://ascl.net/ http://ascl.net/
- eli 9y ago
  To be fair: "All computer codes involved in the creation or analysis of data must also be available to any reader of Science." And "Upon publication, Nature Journals consider it best practice to release custom computer code in a way that allows readers to repeat the published results."
  - jcrites 9y ago
    Could we take it one step further? I'd love to see a publication format that bundles the code, the data, and maybe even the paper together into a single executable bundle for distribution. Imagine that the bundle starts with a manifest file that describes the data and the code, pointing to an authoritative, versioned copy. The user can "run" the manifest, which will download and install the data and any necessary dependencies. (If you're thinking about this from a containerized point of view, the code dependencies and data may be modeled as a container or virtual machine image.) The manifest is like a build file that executes the code passing the data as input, in a clear and reproducible way. Anyone running the manifest will get the same results as the researcher did. Because the entirety of the execution environment is versioned and captured by the manifest, including all of the system software, the results are reproducible from the code and data phase onward with zero effort. For bonus points, the manifest would also "build" the paper (e.g. if it's TeX), and would substitute the results of the code execution directly into the paper, i.e. graphs and numbers. You could conceptualize the paper as something like a notebook, where its values can change dynamically according with the data you provide it.
    - jackcosgrove 9y ago
      Sounds like a Jupyter Notebook.
    - godelmachine 9y ago
      Would all this be possible using git? In that case, even normal people (non-professors , non-R&D engineers) who read research papers to quench their curiosity, would have to know to use git.
      - 9y ago
X86BSD 9y ago
Just curious, do you mind name dropping some of the journals you prefer, for those of us that agree with your view and would like to read them?
- icelancer 9y ago
  I really enjoy PeerJ and their mission. They lack a Sports Science journal which severely limits what I can publish, but I find my way into it via other disciplines when relevant. I help them out as much as possible.

osrec 9y ago

Completely agree with you. I find it especially annoying when research funded by tax payers' money is not made freely available.

avip 9y ago

Don't forget code! "we then used our in-house untested code, developed by an undergrad we scrapped from Biology, to normalize the results. The code is comfortably not attached"

kazinator 9y ago
And that's just the CS papers.
- ziotom78 9y ago
  Not entirely true: I am an astrophysicist, and I can say that this practice would be extremely useful for my discipline as well. In cosmology (my field) we use codes that are often extremely complex, yet they are not usually released together with the papers based on their outputs. This puzzles me, as last year I attended a conference on informatics and astrophysics (ADASS), and one of the talks showed that releasing your codes as open source increases a lot the chance of your papers being cited (expecially if you advertise your code using services like ASCL [1]). The only reason I can think of this unwillingness to publish codes is the fact that these codes written by physicists are often extremely unreadable: very long Fortran routines with awkward variable names, no comments, no test cases, no documentation… Once you get a result from your own software, you get more satisfaction if you publish a paper with the results than if you polish and publish the code. IMHO this is bad science, but it is difficult to change this way of working: cosmology is today done by large collaborations, not by individuals, and if you propose to make the codebase developed by your team public, this idea is usually not welcomed by your co-authors. [1] http://ascl.net/ http://ascl.net/
eli 9y ago
To be fair: "All computer codes involved in the creation or analysis of data must also be available to any reader of Science." And "Upon publication, Nature Journals consider it best practice to release custom computer code in a way that allows readers to repeat the published results."
- jcrites 9y ago
  Could we take it one step further? I'd love to see a publication format that bundles the code, the data, and maybe even the paper together into a single executable bundle for distribution. Imagine that the bundle starts with a manifest file that describes the data and the code, pointing to an authoritative, versioned copy. The user can "run" the manifest, which will download and install the data and any necessary dependencies. (If you're thinking about this from a containerized point of view, the code dependencies and data may be modeled as a container or virtual machine image.) The manifest is like a build file that executes the code passing the data as input, in a clear and reproducible way. Anyone running the manifest will get the same results as the researcher did. Because the entirety of the execution environment is versioned and captured by the manifest, including all of the system software, the results are reproducible from the code and data phase onward with zero effort. For bonus points, the manifest would also "build" the paper (e.g. if it's TeX), and would substitute the results of the code execution directly into the paper, i.e. graphs and numbers. You could conceptualize the paper as something like a notebook, where its values can change dynamically according with the data you provide it.
  - jackcosgrove 9y ago
    Sounds like a Jupyter Notebook.
  - godelmachine 9y ago
    Would all this be possible using git? In that case, even normal people (non-professors , non-R&D engineers) who read research papers to quench their curiosity, would have to know to use git.
    - 9y ago

X86BSD 9y ago

Just curious, do you mind name dropping some of the journals you prefer, for those of us that agree with your view and would like to read them?

icelancer 9y ago
I really enjoy PeerJ and their mission. They lack a Sports Science journal which severely limits what I can publish, but I find my way into it via other disciplines when relevant. I help them out as much as possible.