3 ms·

> It's already difficult to do if your subfield is a different niche. It's not that difficult, at least in science and science-adjacent fields. The scientific

> It's already difficult to do if your subfield is a different niche. It's not that difficult, at least in science and science-adjacent fields. The scientific method still follows the same principles. If you are used to reading papers, you can tell when someone has elided important detail, or not included the evidence for their claims - or, if someone [or a whole subfield] is using technical language to obfuscate their work.

bonoboTP 5y ago
There are many levels of analysis to a paper. Yours is part of it but not the full picture. One other thing is the novelty aspect. To assess that, you need to know the current state of the field. Also the best practices, known methodological pitfalls, judge what is a valid chain of reasoning and what is too speculative etc. It's far from simple and I often see outsiders online criticizing largely irrelevant things while missing core issues. There's no shortcut to expertise in a field. You need first hand experience and knowledge of the state of the art. Otherwise, if papers have to talk to laypeople or generic scientists, we'll see even more smokes and mirrors. The idea of peer review is that a work should be challenged by those best equipped to challenge it.
- teachingassist 5y ago
  A journal could follow a set of statistical standards that could be examined and approved by experts outside of its narrow field. I don't claim that paper reviews have to be or can be perfect - only that they could be held to a better standard than they are now. We see plenty of avenues for publishing to a worse standard (self-publishing on an internet aggregator, for example). Let's see some avenues for a better standard. Why isn't this a science in itself? We could trial some different standards and statistically compare them. [It was not my claim that scientific papers must be understandable by laypeople, although I think this is an ideal case. If not, it should at least be understandable by people in an adjacent subfield - otherwise, it's just a small group of people talking amongst themselves]
Blikkentrekker 5y ago
> The scientific method still follows the same principles. “the scientific method” is rather loosely defined and wide. What passes as “the scientific method” in some fields would not in others.

bonoboTP 5y ago

There are many levels of analysis to a paper. Yours is part of it but not the full picture. One other thing is the novelty aspect. To assess that, you need to know the current state of the field. Also the best practices, known methodological pitfalls, judge what is a valid chain of reasoning and what is too speculative etc. It's far from simple and I often see outsiders online criticizing largely irrelevant things while missing core issues. There's no shortcut to expertise in a field. You need first hand experience and knowledge of the state of the art. Otherwise, if papers have to talk to laypeople or generic scientists, we'll see even more smokes and mirrors. The idea of peer review is that a work should be challenged by those best equipped to challenge it.

teachingassist 5y ago
A journal could follow a set of statistical standards that could be examined and approved by experts outside of its narrow field. I don't claim that paper reviews have to be or can be perfect - only that they could be held to a better standard than they are now. We see plenty of avenues for publishing to a worse standard (self-publishing on an internet aggregator, for example). Let's see some avenues for a better standard. Why isn't this a science in itself? We could trial some different standards and statistically compare them. [It was not my claim that scientific papers must be understandable by laypeople, although I think this is an ideal case. If not, it should at least be understandable by people in an adjacent subfield - otherwise, it's just a small group of people talking amongst themselves]

Blikkentrekker 5y ago

> The scientific method still follows the same principles. “the scientific method” is rather loosely defined and wide. What passes as “the scientific method” in some fields would not in others.