5 ms·
It's even worse than that. The whole peer-review process we have today is stuck in the 19th century. The whole idea that publication and the credibility that
by lisper 7y ago
It's even worse than that. The whole peer-review process we have today is stuck in the 19th century. The whole idea that publication and the credibility that is associated with it should be a binary quality -- a paper is either "published in a peer-reviewd publication", and therefore credible, or it isn't and so it isn't -- is absurd on its face. If there are any shades of grey, they are associated with the "prestige" of the publication, which again, if you think about it, is absurd, and flies in the face of the scientific ideal: an idea is either supported by the data or it isn't, and what two randomly selected anonymous referees on the review board of a journal, people who likely have ulterior motives in terms of their own career advancement, have to say about it should be the least of anyone's concern.
The right model IMHO is open publication a la arxiv with an associated open review process so that anyone who wants to can weigh in. The wheat can be separated from the chaff by doing a "reviewer-rank" analogous to page rank so that the opinions of the people who know what they're doing count more than the crackpots. But this idea that there's an elite priesthood of anointed Scientists, a.k.a. corporate researchers and tenure-track university faculty, who pass final judgement on all scientific truth really has to end.
(And I say this as someone who made their living working successfully within the current system for 15 years. I quit in no small measure because I saw firsthand just how corrupt and dysfunctional it could become, and I was working in a field that is generally considered "hard" science!)
- deleted 7y ago[deleted]
- iagovar 7y agoYou are right, but the parent is on point. When I was a student I tried many times to download datasets, with very little success. Only statistical institutions like INE, Igape and so on happily provided the data used for their studies, which is a given since they publish most of their datasets in their website anyway. This sucks, because maybe you have more questions that could be answered with their datasets, but you can't do shit because this shitty system. I'm not a scientist, and partly because though the uni I was really put of by the bureaucracy and the smell of endogamy, obscure and old system and so on. It's so hard to build new knowledge when everyone is taking care of their little yard and keeping everyone else away... and social sciences need this openness like farmers need their rain.
- lisper 7y ago> You are right, but the parent is on point. Yes, I never meant to imply otherwise.
- sampo 7y ago> The whole peer-review process we have today is stuck in the 19th century. There are journals that do light initial screening, then publish the submitted manuscripts online, then publish the whole peer review (reviewer comments and authors' responses) online, and then proceed (or not) to publish the reviewed article. Also anyone on the internet can leave an additional review, or just some questions, during the review process. Also there are journals that have comment sections for the published articles.
- lisper 7y agoExample(s)?
- sampo 7y agoOpen peer review process: All European Geosciences Union journals https://www.egu.eu/publications/open-access-journals/ https://www.egu.eu/publications/open-access-journals/ Comment section for published articles: For example all PLOS journals https://journals.plos.org/ https://journals.plos.org/
- rrss 7y ago19th century peer review was far more entertaining and public than the current system. Frequently, an article would be published, then various colorful responses would be published tearing apart the article (and not infrequently the character and competence of its authors).
- mherdeg 7y agoThis still happens but it's a lot less fun, yes. See for example "Tai's Formula Is the Trapezoidal Rule" (https://doi.org/10.2337/diacare.17.10.1224 https://doi.org/10.2337/diacare.17.10.1224) and the paper it was responding to, "A Mathematical Model for the Determination of Total Area Under Glucose Tolerance and Other Metabolic Curves" (https://doi.org/10.2337/diacare.17.2.152 https://doi.org/10.2337/diacare.17.2.152)
- thaumasiotes 7y ago> The whole peer-review process we have today is stuck in the 19th century. Peer review as something associated with getting a paper published dates from the mid-20th century. The 19th century used the superior approach of publishing whatever and letting interested parties review it then.
- rjf72 7y agoSocrates and Galileo are but two of the more recognizable names on an endlessly long list. That list being made up of individuals who were correct but had their views judged by the masses, and lost. Socrates was killed, Galileo was merely imprisoned in his house until his final breath. Science is an issue, like many others, that often struggles when you get into matters of popular consensus. New ideas which end up being correct after often vigorously fought against, particularly when they undermine science that others want to be true. We wanted the Earth to be the center of the universe. When somebody published writings suggesting we're just another body of no uniqueness revolving around a similarly indistinct star it posed a typical issue. If a scientist wanted to advance their career, they could come out quite strongly against this. Coming out in support of it risked turn the mob's attention on you. Refuting invalid refutations of this view is a non-trivial matter. Refuting what might, in modern times, be thousands or more - would be impossible. Indeed Galileo's correct heliocentric view would not come to be accepted until many decades after his death. Of course we've overcome such reactionary barbarism though, right? Today there are at least two major issues we face. The first is corporate science. To give a now recently resolved example consider leaded fuel. Leaded fuel is something that from its earliest day drew skepticism. Nonetheless it persisted for the better part of a century with government and scientific endorsements of its safety. This was in large part because the corporations profiting from it made sure to stack the science in their favor and to also vigorously attack any critics of its safety. In an open system you'd see the same problem as above. Support the science suggesting leaded fuel may be unsafe, face the well funded mob. Oppose the science, get grants, accolades, and personal recognition. Compare Midgley, who lives in sufficient infamy that only his last name need be referenced, to e.g. Alice Hamilton. The other issue is one of social matters, very similar to the issues that Galileo faced. Genetics is a perfect modern example. As but one example there, the connection between genetics and intelligence is in no way ambiguous. But this is something we, socially, do not want to accept and so progress is absolutely glacial and if one steps a hair outside an invisible line that we've drawn, they risk getting condemnation, hate, and exclusion from the scientific system. In many ways this is exactly what happened to Galileo. He was not imprisoned for his views, so much as for his disregard for social norms when expressing them - similar as well to the story of Socrates, who was unrepentantly provocative til the end. The problem there is that science should be judged based on merit, not on presentation or nicety. As you move more towards a social system I expect we'd begin to move ever more from the former to the latter. This would be magnified a million times over if we chose to use a pagerank style system as you proposed. I fully agree our current system is, at best, dysfunctional. But I think our idealization of social systems is quite often contradicted by what they turn into - something Socrates, quite prophetically, often spoke of.
- xorand 7y agoNo ranking replaces the access to the real work behind a scientific article: data, programs, experimental procedures, etc. Peer-review is a good thing compared with no review, but access to the real data is simply better. Think that reviewers themselves don't have access to more data than other readers, in the actual system.