4 ms·

But how do I do that? I can interpret a paper and apply it to my problems, but I'm not qualified to tell if it's a good paper[1]. In my field -- computing -- re

But how do I do that? I can interpret a paper and apply it to my problems, but I'm not qualified to tell if it's a good paper[1]. In my field -- computing -- research papers are often written in a language that only just makes sense to me. I'm sure it'd be easier if I was a computer scientist that regularly published, but that's why I suggest you'd need a team of actual scientists who are used to reading, writing, and reviewing papers. As Kernighan almost wrote: Reviewing the paper is twice as hard as reading the paper in the first place. Therefore, if you have to be as clever as possible to read the paper, you are, by definition, not smart enough to review it. [1] Although I can tell whether the code works, the quality of a paper is also about how well-written it is, whether the arguments are valid and supported by the data, and whether it provides something future research can be built on. A terrible paper that happens to contain code that works is useful for me, not so much for science in general.

dalke 10y ago
I believe tomp's suggestion is that you can bootstrap as an overlay journal (see https://en.wikipedia.org/wiki/Overlay_journal https://en.wikipedia.org/wiki/Overlay_journal ) to build up that expertise. You don't need to have experienced people from the start, because they can learn while doing. At the start, no one is going to pay attention to the journal. Then as it gets known for being a good curator, people will start requesting to be reviewed for inclusion. Poof you've got a journal. You might even start with only yourself. And in 5 years if you want to distance yourself from your earlier, naïve efforts, then simply announce a name change. Here's the ugly truth - it's hard to tell if most papers are "good", and there are many types of "good".
- chris_wot 10y ago
  Distancing yourself from early failures is never a good strategy. Too often we flee from our past failures. For me, I don't shy from them but admit they occurred, and that I have learned from them since.
  - dalke 10y ago
    Perhaps I used the term incorrectly. I don't mean "distance" to require someone to "flee" nor not admit its connection, nor do I regard naïvety as a failure. In my field, the main journal used to be "Journal of Computer Documentation". It spun off from "American Documentation" in the early 1960s. The JCD title changed in the 1970s to "Journal of Chemical Information and Computer Sciences" because "documentation" - a 1920s term - was considered already old-fashioned by the 1960s, and the title didn't capture the new focus on computers, and the underlying principles of information science. It then became "Journal of Chemical Information and Modeling" in 2005 because the 1990s showed how to apply those information techniques to make predictive models. Each name change is a form of distancing itself from its previous focus as part of a shift to a new focus. Or is "distance oneself" not applicable for that situation?
    - chris_wot 10y ago
      I think I probably got my wires crossed. By "distancing" I did indeed think you meant "pretend as best as possible previous work never happened in an attempt to protect your reputation", but that's not at all what you meant! Sorry about that.
- mattkrause 10y ago
  Do these actually exist? I would love a "This Week in Vision Research" that has 1-2 paragraph summaries of recent papers, maybe with some attempts at critiques and placing the results in context. Faculty of 1000 is the only thing I know of that's even close to this.
  - dalke 10y ago
    "Discrete Analysis" - http://www.nature.com/news/leading-mathematician-launches-arxiv-overlay-journal-1.18351 http://www.nature.com/news/leading-mathematician-launches-ar... just started and was in the news http://processalgebra.blogspot.com/2015/12/the-novelty-of-arxiv-overlay-journals.html http://processalgebra.blogspot.com/2015/12/the-novelty-of-ar... lists a few more, including "Logical Methods in Computer Science published its first issue ten years ago and has become one of the favourite publication outlets for researchers working on logic in computer science, broadly construed."
tomp 10y ago
Right, you probably can't do it yourself. Even established journals often have a group of peer reviewers etc. Probably the best idea is for a group of academics to get together and start "sharing" the papers they found useful, acting as a "peer review" group of sorts. I was just commenting about how to get around the "the journal isn't prestigious so no important research is published in it so it's not prestigious" problem.

dalke 10y ago

I believe tomp's suggestion is that you can bootstrap as an overlay journal (see https://en.wikipedia.org/wiki/Overlay_journal https://en.wikipedia.org/wiki/Overlay_journal ) to build up that expertise. You don't need to have experienced people from the start, because they can learn while doing. At the start, no one is going to pay attention to the journal. Then as it gets known for being a good curator, people will start requesting to be reviewed for inclusion. Poof you've got a journal. You might even start with only yourself. And in 5 years if you want to distance yourself from your earlier, naïve efforts, then simply announce a name change. Here's the ugly truth - it's hard to tell if most papers are "good", and there are many types of "good".

chris_wot 10y ago
Distancing yourself from early failures is never a good strategy. Too often we flee from our past failures. For me, I don't shy from them but admit they occurred, and that I have learned from them since.
- dalke 10y ago
  Perhaps I used the term incorrectly. I don't mean "distance" to require someone to "flee" nor not admit its connection, nor do I regard naïvety as a failure. In my field, the main journal used to be "Journal of Computer Documentation". It spun off from "American Documentation" in the early 1960s. The JCD title changed in the 1970s to "Journal of Chemical Information and Computer Sciences" because "documentation" - a 1920s term - was considered already old-fashioned by the 1960s, and the title didn't capture the new focus on computers, and the underlying principles of information science. It then became "Journal of Chemical Information and Modeling" in 2005 because the 1990s showed how to apply those information techniques to make predictive models. Each name change is a form of distancing itself from its previous focus as part of a shift to a new focus. Or is "distance oneself" not applicable for that situation?
  - chris_wot 10y ago
    I think I probably got my wires crossed. By "distancing" I did indeed think you meant "pretend as best as possible previous work never happened in an attempt to protect your reputation", but that's not at all what you meant! Sorry about that.
mattkrause 10y ago
Do these actually exist? I would love a "This Week in Vision Research" that has 1-2 paragraph summaries of recent papers, maybe with some attempts at critiques and placing the results in context. Faculty of 1000 is the only thing I know of that's even close to this.
- dalke 10y ago
  "Discrete Analysis" - http://www.nature.com/news/leading-mathematician-launches-arxiv-overlay-journal-1.18351 http://www.nature.com/news/leading-mathematician-launches-ar... just started and was in the news http://processalgebra.blogspot.com/2015/12/the-novelty-of-arxiv-overlay-journals.html http://processalgebra.blogspot.com/2015/12/the-novelty-of-ar... lists a few more, including "Logical Methods in Computer Science published its first issue ten years ago and has become one of the favourite publication outlets for researchers working on logic in computer science, broadly construed."

tomp 10y ago

Right, you probably can't do it yourself. Even established journals often have a group of peer reviewers etc. Probably the best idea is for a group of academics to get together and start "sharing" the papers they found useful, acting as a "peer review" group of sorts. I was just commenting about how to get around the "the journal isn't prestigious so no important research is published in it so it's not prestigious" problem.