3 ms·

> The paper is not the final word on anything anyway. If you read a paper expecting to learn something about the world, you are doing something wrong. Can you

> The paper is not the final word on anything anyway. If you read a paper expecting to learn something about the world, you are doing something wrong. Can you expand on this? As a layperson, I'm wondering what would you read a paper for at all if not to learn something about the world? What's the point otherwise?

jltsiren 4y ago
Papers report very early stages of knowledge formation. They typically contain an argument: what was done, why, what was expected to happen, what actually happened, how did the authors interpret that, and why. Rather than describing the world directly, a paper describes a belief about the world, and justifications for the belief. Published results are often contradictory, because individual papers are unreliable. Something may have gone subtly wrong, the interpretation could lack nuances, some key understanding may still be missing, or the authors may have just been unlucky. When an expert reads many papers on related topics, the arguments shape their beliefs. Eventually a scientific consensus may emerge, which is the next (but still an insufficient) step towards reliable knowledge.
- tptacek 4y ago
  This is something I bookmarked (in part because I'm an admirer, but also because it made sense and squared with my limited experience of academic peer review): https://twitter.com/TheSavageInMan/status/1083503677157961728 https://twitter.com/TheSavageInMan/status/108350367715796172... I think people get in trouble when they think academic publication has value in itself. Ideas become valuable when people care about them. Publication is one particular path to reach a community of readers but it doesn’t make your ideas matter.
  - ifyoubuildit 4y ago
    Isn't this a kinda narrow view of what's valuable? Sure, it's valuable for your reputation or your academic career only if it becomes widespread. But if something has predictive power, it's valuable no matter how many other people know it. In some cases, it's way more valuable when it is still unknown, because nobody has had the chance to capitalize on it yet.
    - tptacek 4y ago
      I think you're talking about science in general, and Stefan is talking specifically about papers.
      - ifyoubuildit 4y ago
        I'm not familiar with the person behind the tweet, so I don't know any more than what the quote says. But I'm saying that if an individual paper says a thing, and that thing ends up being useful, then how is that not valuable? It could end up being wrong, but so could anything else you pick up from anywhere. In some instances, it won't end up being useful because it's only relevant to someone in a lab full of million dollar equipment. But that'll only be the case sometimes.
        tptacek 4y ago
        Some papers are super important! Some papers aren't. (from the ice age... to the dole age...) "Being a paper" isn't a very meaningful metric. That's all that's being said.
- ifyoubuildit 4y ago
  I think this is a problem if you take one paper and then you weight its conclusion at 100%. It's less so if you take all of the conclusions and realize they all have a weight (maybe 0). But I think that's true for every source of information. My approach is to avoid framing things in terms of "now I know x is true", and instead look at it in terms of "someone believes (or wants me to believe) x is true". I then weigh all of those beliefs as I observe the world around me. I resign myself to never really knowing anything myself, but having some idea of what different schools of thought are on a given topic. If you move your information gathering further along the chain, to maybe a text book, or some expert's twitter feed, why should these things be more reliable? If they've correctly come upon the consensus view, you still have to consider that informing you of the truth night not be their first priority. They could be after money, advancement in their field, political agendas, and (though less likely) they still could be just plain wrong. So they're more likely to know the truth, but you're still unable to evaluate whether they're giving it to you or not.

jltsiren 4y ago

Papers report very early stages of knowledge formation. They typically contain an argument: what was done, why, what was expected to happen, what actually happened, how did the authors interpret that, and why. Rather than describing the world directly, a paper describes a belief about the world, and justifications for the belief. Published results are often contradictory, because individual papers are unreliable. Something may have gone subtly wrong, the interpretation could lack nuances, some key understanding may still be missing, or the authors may have just been unlucky. When an expert reads many papers on related topics, the arguments shape their beliefs. Eventually a scientific consensus may emerge, which is the next (but still an insufficient) step towards reliable knowledge.

tptacek 4y ago
This is something I bookmarked (in part because I'm an admirer, but also because it made sense and squared with my limited experience of academic peer review): https://twitter.com/TheSavageInMan/status/1083503677157961728 https://twitter.com/TheSavageInMan/status/108350367715796172... I think people get in trouble when they think academic publication has value in itself. Ideas become valuable when people care about them. Publication is one particular path to reach a community of readers but it doesn’t make your ideas matter.
- ifyoubuildit 4y ago
  Isn't this a kinda narrow view of what's valuable? Sure, it's valuable for your reputation or your academic career only if it becomes widespread. But if something has predictive power, it's valuable no matter how many other people know it. In some cases, it's way more valuable when it is still unknown, because nobody has had the chance to capitalize on it yet.
  - tptacek 4y ago
    I think you're talking about science in general, and Stefan is talking specifically about papers.
    - ifyoubuildit 4y ago
      I'm not familiar with the person behind the tweet, so I don't know any more than what the quote says. But I'm saying that if an individual paper says a thing, and that thing ends up being useful, then how is that not valuable? It could end up being wrong, but so could anything else you pick up from anywhere. In some instances, it won't end up being useful because it's only relevant to someone in a lab full of million dollar equipment. But that'll only be the case sometimes.
      - tptacek 4y ago
        Some papers are super important! Some papers aren't. (from the ice age... to the dole age...) "Being a paper" isn't a very meaningful metric. That's all that's being said.
ifyoubuildit 4y ago
I think this is a problem if you take one paper and then you weight its conclusion at 100%. It's less so if you take all of the conclusions and realize they all have a weight (maybe 0). But I think that's true for every source of information. My approach is to avoid framing things in terms of "now I know x is true", and instead look at it in terms of "someone believes (or wants me to believe) x is true". I then weigh all of those beliefs as I observe the world around me. I resign myself to never really knowing anything myself, but having some idea of what different schools of thought are on a given topic. If you move your information gathering further along the chain, to maybe a text book, or some expert's twitter feed, why should these things be more reliable? If they've correctly come upon the consensus view, you still have to consider that informing you of the truth night not be their first priority. They could be after money, advancement in their field, political agendas, and (though less likely) they still could be just plain wrong. So they're more likely to know the truth, but you're still unable to evaluate whether they're giving it to you or not.