3 ms·

I don't think peer review is about ensuring anything. It's about spending some time to improve the quality of the paper to save time for everyone else. It's mor

I don't think peer review is about ensuring anything. It's about spending some time to improve the quality of the paper to save time for everyone else. It's more about providing editorial services than about the reported work itself. A research paper is supposed to be a honest report of best efforts to study a topic. If it's not, that's a problem that can't be solved by having a few people spending a few hours with the report. The paper is not the final word on anything anyway. If you read a paper expecting to learn something about the world, you are doing something wrong. As a reviewer, you determine whether the paper is interesting and relevant to the venue. You report any issues you spot that should be corrected and any things you believe that could be improved. And if you get any ideas you feel like sharing, you may share them as well. And that's it.

ifyoubuildit 4y ago
> The paper is not the final word on anything anyway. If you read a paper expecting to learn something about the world, you are doing something wrong. Can you expand on this? As a layperson, I'm wondering what would you read a paper for at all if not to learn something about the world? What's the point otherwise?
- jltsiren 4y ago
  Papers report very early stages of knowledge formation. They typically contain an argument: what was done, why, what was expected to happen, what actually happened, how did the authors interpret that, and why. Rather than describing the world directly, a paper describes a belief about the world, and justifications for the belief. Published results are often contradictory, because individual papers are unreliable. Something may have gone subtly wrong, the interpretation could lack nuances, some key understanding may still be missing, or the authors may have just been unlucky. When an expert reads many papers on related topics, the arguments shape their beliefs. Eventually a scientific consensus may emerge, which is the next (but still an insufficient) step towards reliable knowledge.
  - tptacek 4y ago
    This is something I bookmarked (in part because I'm an admirer, but also because it made sense and squared with my limited experience of academic peer review): https://twitter.com/TheSavageInMan/status/1083503677157961728 https://twitter.com/TheSavageInMan/status/108350367715796172... I think people get in trouble when they think academic publication has value in itself. Ideas become valuable when people care about them. Publication is one particular path to reach a community of readers but it doesn’t make your ideas matter.
    - ifyoubuildit 4y ago
      Isn't this a kinda narrow view of what's valuable? Sure, it's valuable for your reputation or your academic career only if it becomes widespread. But if something has predictive power, it's valuable no matter how many other people know it. In some cases, it's way more valuable when it is still unknown, because nobody has had the chance to capitalize on it yet.
      - tptacek 4y ago
        I think you're talking about science in general, and Stefan is talking specifically about papers.
        ifyoubuildit 4y ago
        I'm not familiar with the person behind the tweet, so I don't know any more than what the quote says. But I'm saying that if an individual paper says a thing, and that thing ends up being useful, then how is that not valuable? It could end up being wrong, but so could anything else you pick up from anywhere. In some instances, it won't end up being useful because it's only relevant to someone in a lab full of million dollar equipment. But that'll only be the case sometimes.
        tptacek 4y ago
        Some papers are super important! Some papers aren't. (from the ice age... to the dole age...) "Being a paper" isn't a very meaningful metric. That's all that's being said.

ifyoubuildit 4y ago

> The paper is not the final word on anything anyway. If you read a paper expecting to learn something about the world, you are doing something wrong. Can you expand on this? As a layperson, I'm wondering what would you read a paper for at all if not to learn something about the world? What's the point otherwise?

jltsiren 4y ago
Papers report very early stages of knowledge formation. They typically contain an argument: what was done, why, what was expected to happen, what actually happened, how did the authors interpret that, and why. Rather than describing the world directly, a paper describes a belief about the world, and justifications for the belief. Published results are often contradictory, because individual papers are unreliable. Something may have gone subtly wrong, the interpretation could lack nuances, some key understanding may still be missing, or the authors may have just been unlucky. When an expert reads many papers on related topics, the arguments shape their beliefs. Eventually a scientific consensus may emerge, which is the next (but still an insufficient) step towards reliable knowledge.
- tptacek 4y ago
  This is something I bookmarked (in part because I'm an admirer, but also because it made sense and squared with my limited experience of academic peer review): https://twitter.com/TheSavageInMan/status/1083503677157961728 https://twitter.com/TheSavageInMan/status/108350367715796172... I think people get in trouble when they think academic publication has value in itself. Ideas become valuable when people care about them. Publication is one particular path to reach a community of readers but it doesn’t make your ideas matter.
  - ifyoubuildit 4y ago
    Isn't this a kinda narrow view of what's valuable? Sure, it's valuable for your reputation or your academic career only if it becomes widespread. But if something has predictive power, it's valuable no matter how many other people know it. In some cases, it's way more valuable when it is still unknown, because nobody has had the chance to capitalize on it yet.
    - tptacek 4y ago
      I think you're talking about science in general, and Stefan is talking specifically about papers.
      - ifyoubuildit 4y ago
        I'm not familiar with the person behind the tweet, so I don't know any more than what the quote says. But I'm saying that if an individual paper says a thing, and that thing ends up being useful, then how is that not valuable? It could end up being wrong, but so could anything else you pick up from anywhere. In some instances, it won't end up being useful because it's only relevant to someone in a lab full of million dollar equipment. But that'll only be the case sometimes.
        tptacek 4y ago
        Some papers are super important! Some papers aren't. (from the ice age... to the dole age...) "Being a paper" isn't a very meaningful metric. That's all that's being said.