6 ms·

That is exactly the problem. All this stuff is supposed to be in the paper, the paper is not a press release. And everyone understands that its hard to know ex

by nonbel 8y ago

That is exactly the problem. All this stuff is supposed to be in the paper, the paper is not a press release. And everyone understands that its hard to know exactly what info someone else will be missing, so it wont be perfect at first. That is why there is supposed to be a back and forth the first time a new method is used. Figuring out what needs to go into the methods is going to be an iterative process. As it is right now its a disaster though. Like they say in this article no one even knows the most basic info like cell density. The entire system is not set up to deal with replications at all because they haven't been doing them.

rjkennedy98 8y ago
> All this stuff is supposed to be in the paper, the paper is not a press release. Welcome to the real world. Science doesn't work like it was described in your 7th grade science class.
- pas 8y ago
  But it can be approximated. Repeatable experiments. Done by machines, properly accounted ingredients and so on. Physics does it. It should be perfectly doable with cell cultures too. Mice are of course a lot harder. But if we wouldn't wait decades to replicate, but could simply get in touch with the originators - and so do a collaborative effort in establishing the parameters.
- nonbel 8y ago
  These sayings like "welcome to the real world", "there's a difference between theory and practice", thought-killers are just excuses for being lazy. Science is hard, really hard. People who think doing a good job is too hard should get out of science instead of polluting the literature with their statistically significant results or whatever. Unfortunately its been the opposite happening and the people who cant handle it have been training the new people how to get away with half-assing it. This has been going on for a long time now, at least since the 1940s in some areas...
  - timr 8y ago
    There's a huge difference between "bad science", and "it doesn't live up to the average HN reader's belief of how science works". Maybe the parent's comment is harsh, but the sentiment is, IMO, correct. Science is a system, not a series of published recipes. Even the best, most-cited papers in scientific history, when taken in isolation, are often riddled with problems. No paper is perfect, but the process of science, over time, validates the papers that are worth validating. Experiments are rarely reproduced exactly, but rather, reproduced under a thousand different conditions, thus proving that the result is robust over time. I've often felt that these efforts to mass-reproduce scientific literature are misguided. Anyone who has worked in a lab knows that most papers are difficult to reproduce for any number of reasons, and that even if you have a perfect description of materials and methods, plenty of correct and valid results are difficult to reproduce, just because lab work is hard and messy. It's not fair to fling accusations of impropriety simply because an experiment doesn't work in your hands.
    - JumpCrisscross 8y ago
      > Science is a system, not a series of published recipe Peer review and reproduction are essential components of a scientific system. If a paper does not provide—it its main text or supplements—the necessary detail to enable reproduction, it is not a scientific paper but a press release. > plenty of correct and valid results are difficult to reproduce There is lots of good science in figuring out why certain results reproduce in certain circumstances and not in others. With insufficient methodological detail, this exploration is rendered impossible.
      - timr 8y ago
        "If a paper does not provide—it its main text or supplements—the necessary detail to enable reproduction, it is not a scientific paper but a press release." No. That's ridiculous. When a paper doesn't have all of the necessary details to reproduce, then scientists contact the authors, and try to get those details. In nearly all cases, the details are resolved without incident. If the omissions are sufficiently egregious, errata are issued. If they're really, really bad, papers are withdrawn. If a paper is so old that the original authors are long gone, you look for a citation history, and see who else reproduced the work over the years. Papers that were never reproduced are unlikely to be fruitful. Mistakes happen and details get omitted, but thankfully, we have lots of ways of working around those mistakes. Nobody -- except for armchair scientists -- assumes that scientific knowledge is communicated exclusively by the methods sections of publications.
- p1necone 8y ago
  This isn't helpful. People questioning this is a good thing. We can't just throw our hands in the air and go "welp, shit's fucked, no point in trying to fix it".
chrisseaton 8y ago
> All this stuff is supposed to be in the paper I don't agree. I think this stuff should be in supplemental notes (we call it the 'artefact' in my field, not sure if that's universal). A paper should be reasonably readable, not a huge list of processes in detail.
- nonbel 8y ago
  I'm used to biomed where the "supplements" do exist and are containing more and more of the actual information while the "paper" is becoming more of an "executive summary". Since I find all that info in the same place, I consider it part of "the paper".

by nonbel 8y ago

rjkennedy98 8y ago

> All this stuff is supposed to be in the paper, the paper is not a press release. Welcome to the real world. Science doesn't work like it was described in your 7th grade science class.

pas 8y ago
But it can be approximated. Repeatable experiments. Done by machines, properly accounted ingredients and so on. Physics does it. It should be perfectly doable with cell cultures too. Mice are of course a lot harder. But if we wouldn't wait decades to replicate, but could simply get in touch with the originators - and so do a collaborative effort in establishing the parameters.
nonbel 8y ago
These sayings like "welcome to the real world", "there's a difference between theory and practice", thought-killers are just excuses for being lazy. Science is hard, really hard. People who think doing a good job is too hard should get out of science instead of polluting the literature with their statistically significant results or whatever. Unfortunately its been the opposite happening and the people who cant handle it have been training the new people how to get away with half-assing it. This has been going on for a long time now, at least since the 1940s in some areas...
- timr 8y ago
  There's a huge difference between "bad science", and "it doesn't live up to the average HN reader's belief of how science works". Maybe the parent's comment is harsh, but the sentiment is, IMO, correct. Science is a system, not a series of published recipes. Even the best, most-cited papers in scientific history, when taken in isolation, are often riddled with problems. No paper is perfect, but the process of science, over time, validates the papers that are worth validating. Experiments are rarely reproduced exactly, but rather, reproduced under a thousand different conditions, thus proving that the result is robust over time. I've often felt that these efforts to mass-reproduce scientific literature are misguided. Anyone who has worked in a lab knows that most papers are difficult to reproduce for any number of reasons, and that even if you have a perfect description of materials and methods, plenty of correct and valid results are difficult to reproduce, just because lab work is hard and messy. It's not fair to fling accusations of impropriety simply because an experiment doesn't work in your hands.
  - JumpCrisscross 8y ago
    > Science is a system, not a series of published recipe Peer review and reproduction are essential components of a scientific system. If a paper does not provide—it its main text or supplements—the necessary detail to enable reproduction, it is not a scientific paper but a press release. > plenty of correct and valid results are difficult to reproduce There is lots of good science in figuring out why certain results reproduce in certain circumstances and not in others. With insufficient methodological detail, this exploration is rendered impossible.
    - timr 8y ago
      "If a paper does not provide—it its main text or supplements—the necessary detail to enable reproduction, it is not a scientific paper but a press release." No. That's ridiculous. When a paper doesn't have all of the necessary details to reproduce, then scientists contact the authors, and try to get those details. In nearly all cases, the details are resolved without incident. If the omissions are sufficiently egregious, errata are issued. If they're really, really bad, papers are withdrawn. If a paper is so old that the original authors are long gone, you look for a citation history, and see who else reproduced the work over the years. Papers that were never reproduced are unlikely to be fruitful. Mistakes happen and details get omitted, but thankfully, we have lots of ways of working around those mistakes. Nobody -- except for armchair scientists -- assumes that scientific knowledge is communicated exclusively by the methods sections of publications.
p1necone 8y ago
This isn't helpful. People questioning this is a good thing. We can't just throw our hands in the air and go "welp, shit's fucked, no point in trying to fix it".

chrisseaton 8y ago

> All this stuff is supposed to be in the paper I don't agree. I think this stuff should be in supplemental notes (we call it the 'artefact' in my field, not sure if that's universal). A paper should be reasonably readable, not a huge list of processes in detail.

nonbel 8y ago
I'm used to biomed where the "supplements" do exist and are containing more and more of the actual information while the "paper" is becoming more of an "executive summary". Since I find all that info in the same place, I consider it part of "the paper".