3 ms·

What is an "open" paper ? I mean, the paper is a very succinct description of the research done. For example, if one demonstrates how one builds a CNN but at th

by wiz21c 9y ago

What is an "open" paper ? I mean, the paper is a very succinct description of the research done. For example, if one demonstrates how one builds a CNN but at the same time, doesn't provide the data set used to build it, then it's worthless. In the same vein, if one provides explanations on how to detect motion in a picture but doesn't provide the full source code of its detection pipeline, then it may be super tough to rebuild that code (because it assumes the paper actually gives all the necessary information, not the "general idea")) So, how open is open ?

kuschku 9y ago
This is why about 90% of the papers Google publishes are useless. They describe deep learning successes, with datasets that aren't public, on hardware that isn't public, with software that isn't public. They describe technologies, be it spanner, borg, flumejava or others, relying on closed hard- and software. If an average university professor can't replicate research 1:1 based solely on existing, open technology and the paper, the paper should not be accepted by any journal. In fact, it shouldn't even be accepted on conferences, or be able to get a doi code. That's not worthy of being called science.
- remus 9y ago
  I don't think they're quite useless (they tend to contain some interesting ideas, which presumably work as google have generally implemented them) but as you say they're definitely not 'science' as they can't be replicated.
- dagw 9y ago
  While I obviously agree that fully reproducible results is the ideal, just knowing that something is possible and having some idea where to start is often valuable in itself.
  - kuschku 9y ago
    First of all, if it’s not reproducible, it’s not science. The entire point of science is getting results that give insight to others. Second, often the papers set prerequisites that are not available to anyone else, because these, too, are undocumented. This is cool if you want an inspiration, but large parts of these papers are as useful as watching Star Trek for actual science. They can serve as inspiration, but they can’t tell you how to replicate something, or give you possible leads to expand on it. And this issue gets even worse – if I’m writing an undergrad thesis on a tech that Google has several years lead on, my thesis won’t be able to produce anything new. I can’t expand on the state of the art, because the state of the art isn’t even available to me.
Vinnl 9y ago
The Open Access movement (making articles openly available) is a smaller part of the larger Open Science movement, which encompasses among others making code and data open, registering research plans in advance, publishing null results, etc.

by wiz21c 9y ago

kuschku 9y ago

This is why about 90% of the papers Google publishes are useless. They describe deep learning successes, with datasets that aren't public, on hardware that isn't public, with software that isn't public. They describe technologies, be it spanner, borg, flumejava or others, relying on closed hard- and software. If an average university professor can't replicate research 1:1 based solely on existing, open technology and the paper, the paper should not be accepted by any journal. In fact, it shouldn't even be accepted on conferences, or be able to get a doi code. That's not worthy of being called science.

remus 9y ago
I don't think they're quite useless (they tend to contain some interesting ideas, which presumably work as google have generally implemented them) but as you say they're definitely not 'science' as they can't be replicated.
dagw 9y ago
While I obviously agree that fully reproducible results is the ideal, just knowing that something is possible and having some idea where to start is often valuable in itself.
- kuschku 9y ago
  First of all, if it’s not reproducible, it’s not science. The entire point of science is getting results that give insight to others. Second, often the papers set prerequisites that are not available to anyone else, because these, too, are undocumented. This is cool if you want an inspiration, but large parts of these papers are as useful as watching Star Trek for actual science. They can serve as inspiration, but they can’t tell you how to replicate something, or give you possible leads to expand on it. And this issue gets even worse – if I’m writing an undergrad thesis on a tech that Google has several years lead on, my thesis won’t be able to produce anything new. I can’t expand on the state of the art, because the state of the art isn’t even available to me.

Vinnl 9y ago

The Open Access movement (making articles openly available) is a smaller part of the larger Open Science movement, which encompasses among others making code and data open, registering research plans in advance, publishing null results, etc.