4 ms·
Stefan Savage put it best, I think: a paper accepted in a journal is part of a conversation that science is having; it's the start of a debate, not the conclusi
by tptacek 3y ago
Stefan Savage put it best, I think: a paper accepted in a journal is part of a conversation that science is having; it's the start of a debate, not the conclusion. What's important about a paper is whether the ideas in it are validated and get built on by other scientists. Peer review is just a sanity check before that process starts, nothing more.
- HarryHirsch 3y agoWell, yes. But what's worrying is that so many people who have seen a university from inside and even have advanced degrees do not know this. Every time some one comes up with grants for reproducing published stuff the silly idea finds enthusiastic support. How come? People should know about the practice of science but don't.
- BalinKing 3y agoIt seems like you're agreeing with the parent, but then the sentence > Every time some one comes up with grants for reproducing published stuff the silly idea finds enthusiastic support. suggests that you think replication isn't useful. Or, am I misunderstanding what "the silly idea" refers to?
- wizzwizz4 3y agoThe silly idea is the thing they're trying to replicate (likely out of suspicion that it's a false silly idea).
- j7ake 3y agoThe value of a paper depends on much it influences the thinking of other scientists multiplied by the number of scientists it influences. Therefore the recent super conductivity papers may well end up being very important, if only to stir up the community to action.
- loceng 3y agoI think us humans are flawed enough where we need a reminder of timescale, e.g. how time tested is a theory, how long did a different belief exist before a new understanding arose? Einstein for example had public resistance to his theory of relativity in the beginning.
- dctoedt 3y ago> Einstein for example had public resistance to his theory of relativity in the beginning. More recently, it took years for the medical community to get over its opposition to the evidence that common peptic ulcers are caused by bacteria (which earned two Aussie physicians the 2005 Nobel Prize in Medicine). [0] A NY Times correspondent (and physician) wrote in 2002, "I’ve never seen the medical community more defensive or more critical of a story." (Much of the opposition was interest-based, e.g., surgeons who didn't want to lose lucrative fees for operating on ulcer patients who could instead be treated with inexpensive antibiotics without the pain and recovery time of major surgery.) [0] https://www.nobelprize.org/prizes/medicine/2005/7693-the-nobel-prize-in-physiology-or-medicine-2005-2005-6/ https://www.nobelprize.org/prizes/medicine/2005/7693-the-nob... [1] https://en.wikipedia.org/wiki/Timeline_of_peptic_ulcer_disease_and_Helicobacter_pylori#1970%E2%80%9321st_century https://en.wikipedia.org/wiki/Timeline_of_peptic_ulcer_disea...
- loceng 3y agoIt's only relatively recently that gut biome health can be a major contributing factor to mental health, due to the neurotransmitters produced in the gut: https://www.google.com/search?q=biome+gut+health+neurotransmitters https://www.google.com/search?q=biome+gut+health+neurotransm...
- photochemsyn 3y agoA funny story from academia is about the cohort of peer reviewers whose primary standard for acceptance is that their own work is cited in the bibliography of the submitted paper. Peer review often fails to catch cheaters, too, as the infamous Jan Hendrik Schön case demonstrated (from 2000-2001 the Bell Labs researcher published 9 papers in Science and 7 in Nature, all of which made it through peer review, all of which were later retracted on grounds of fraudulent data manipulation). In the long run, the scandal did improve the field, as all publications on microelectronic graphite etc. devices now require electron microscopy proof that the claimed devices actually exist. Note current AI technology allows for data fraud and image manipulation that's much harder to detect than in the past, though.
- spookie 3y agoYes, this is indeed a big problem. If anything, more eyes are needed. Therefore, less walls.
- tptacek 3y agoIn a lot of fields, more eyes aren't going to create fewer walls, because (a) the important venues are selecting K winners out of N submissions, and that K is the bottleneck, and (b) in most fields, any real (and some unreal) results will get published somewhere.
- neilv 3y ago> A funny story from academia is about the cohort of peer reviewers whose primary standard for acceptance is that their own work is cited in the bibliography of the submitted paper. In some circles, the two first pieces of feedback on a paper draft, from a nominal co-author: 1. Cite [my people's various loosely related work]. 2. Cite [particular researcher in this niche], who'll probably be a reviewer.
- RcouF1uZ4gsC 3y agoThen devalue published papers. Don’t use them as the basis for rewards. Don’t use them as the basis for policy If they are nothing more than the the starting conversation of scientists then we should not put much value in them. Let’s not play the have it both ways game where scientific papers are given deference and then when there is cheating just say li g that scientific papers are just a conversation starter.
- tptacek 3y agoPeople doing science professionally already understand the value of a paper.
- concordDance 3y agoIf not using papers as a basis for policy then what? Making a committee of academics from a selection of prestigious institutions every time you have a question that needs answering?
- neilv 3y ago> * Then devalue published papers. Don’t use them as the basis for rewards.* What about the use of papers as metrics for advancing in academic careers? Career advancement could be based on market forces (e.g., productive researcher on something valued, if they're not given a sweet enough deal by one university, another university can snap them up). But how do buyers in the market get the information about how valuable a given researcher is? (Word of mouth? CV summary without pubs list? Job talks? Interviews? Twitter?)
- Almondsetat 3y agoWe should start framing some journals as "here's what the authors have found convincing from bleeding edge submissions, please take a look and try to confirm these findings" for other researchers and others as "here's a conservative and comprehensive list of results which have been solidly reproduced" for professionals in the field who want dependable stuff
- zamfi 3y ago> here's a conservative and comprehensive list of results which have been solidly reproduced This sounds more like textbooks. Journals are usually intended to be venues for experts to talk to each other, not even "professionals in the field". There are some journals (sometimes called "translational") that try to bring scientific results into practice, though these are often limited to fields where "practice" is its own huge body of knowledge that doesn't always overlap with "research" in that field (e.g., Medicine).
- treyd 3y ago> Journals are usually intended to be venues for experts to talk to each other, not even "professionals in the field". This made sense when communication was slow and the back and forth of discourse could take weeks to months. But publishing your work in a way that thousands of other readers can consume it is easier now that it's been ever, so it stands to reason that sharing the results of scientific works may yield better outcomes if it takes on a different form.
- tptacek 3y agoThere's a whole Internet to post important results on without waiting for peer review. If you're an established practitioner in your field making claims that are straightforward to follow, other practitioners will probably read you.
- Almondsetat 3y agoHow can it be an important result if nobody has peer reviewed it let alone reproduced it? It's a potentially important claim