5 ms·
Do you really need peer review? Isn’t number of citations a form of peer review? Of course you sort of need to assign a reputation to these citations.
by cm2187 9y ago
Do you really need peer review? Isn’t number of citations a form of peer review? Of course you sort of need to assign a reputation to these citations.
- TheCondor 9y agoYou absolutely need peer review. You’d be stunned at some of the rejected submissions
- barry-cotter 9y agoAbsolute crap will be roundly ignored with or without peer review. Science and the Humanities worked fine before peer review and if it’s dropped they’ll work fine again. If Grigori Perelman puts another paper on arxiv that’s groundbreaking people will look at it without benefit of peer review, just like the last one.
- Mediterraneo10 9y ago> Absolute crap will be roundly ignored with or without peer review. Things might be ignored, but only after a person has already wasted their time looking at it and determining it is crap. It was harder for cranks to publish back when journals were purely physical and a crank would have to come up with the printing costs himself. Now that anyone can publish for free on the internet, peer review is even more important for establishing what content out there is worth paying attention to and which is not.
- barry-cotter 9y agoYou neglect the costs of submission, revision, resubmisssion etc. which are borne by the authors. Even if we assume that all involved are pure of heart and no one is deliberately delaying publication of their rivals’ work submission to publication in economics is on the order of two years. This is insane so the actual intellectual conversation has moved to working papers with final publication being in a journal being as much for archival and career progression metrics as anything else. I believe the situation in much of computer science is similar, with conference papers serving as the workaround for the fact that pre publication peer review is unbearably slow, not working papers. Peer review happens anyway, but faster, and in public, without the insanity of revise and resubmit. Journals are not where the action is in Economics or Computer Science. It works for them. Why not for everybody?
- Mediterraneo10 9y ago> You neglect the costs of submission, revision, resubmisssion etc. which are borne by the authors. I neglect those because journals in my own field are both free to publish in and, more often than not, open-access. I do understand that not everyone is so fortunate, however.
- barry-cotter 9y agoI’m talking about time, not money. Because those costs absolutely are borne by the authors. Insofar as peer review hinders the free dissemination of ideas it also hurts the scientific community and those who depend on its research.
- jclos 9y ago> Science and the Humanities worked fine before peer review Before peer review they were not dealing with the volume of scientific content that is submitted nowadays. One could debate whether citations are the only metric necessary, but there needs to be a filter before that even happens, or else we will be flooded with an avalanche of garbage research. It would be trivial for someone with deep enough pockets to order a network of junk papers citing each other and debunking climate change or evolution and make a real mess. As painful as reviewing is, it's a necessary evil for now.
- FabHK 9y ago> You’d be stunned at some of the rejected submissions You'll be stunned by some of the accepted submissions, such as these highlighted by "New Real Peer Review", @RealPeerReview on Twitter: Glaciers, gender, and science: A feminist glaciology framework for global environmental change research. "Merging feminist postcolonial science studies and feminist political ecology, the feminist glaciology framework generates robust analysis of gender, power, and epistemologies in dynamic social-ecological systems, thereby leading to more just and equitable science and human-ice interactions." http://journals.sagepub.com/doi/abs/10.1177/0309132515623368 http://journals.sagepub.com/doi/abs/10.1177/0309132515623368 Black Anality "In turning attention to this understudied and overdetermining space — the black anus — “Black Anality” considers the racial meanings produced in pornographic texts that insistently return to the black female anus as a critical site of pleasure, peril, and curiosity." https://read.dukeupress.edu/glq/article-abstract/20/4/439/34905/Black-Anality https://read.dukeupress.edu/glq/article-abstract/20/4/439/34... The Perilous Whiteness of Pumpkins (specifically, Starbucks’ pumpkin spice lattes) http://www.tandfonline.com/doi/abs/10.1080/2373566X.2015.1099421?journalCode=rgeo20 http://www.tandfonline.com/doi/abs/10.1080/2373566X.2015.109... EGO HIPPO: the subject as metaphor about a trans animal scholar identifying as a hippopotamus. http://www.tandfonline.com/doi/full/10.1080/0969725X.2017.1322822 http://www.tandfonline.com/doi/full/10.1080/0969725X.2017.13... Rum, rytm och resande: Genusperspektiv på järnvägsstationer PhD thesis on gendered train stations. "The overriding aim of this study is to examine how male and female commuters use and experience railway stations as gendered physical places and social spaces, during their daily travels. [...] Through this theoretical frame the thesis analyses gendered power relations of bodies in time, space and mobility." http://liu.diva-portal.org/smash/record.jsf?pid=diva2%3A742447&dswid=3974 http://liu.diva-portal.org/smash/record.jsf?pid=diva2%3A7424... “I'm a real catch”: The blurring of alternative and hegemonic masculinities in men's talk about home cooking "while many participants drew on what they saw as alternative masculinities to frame their cooking, these masculinities may in fact have hegemonic elements revolving around notions of individuality and romantic or sexual allure." https://www.sciencedirect.com/science/article/abs/pii/S0277539513001453 https://www.sciencedirect.com/science/article/abs/pii/S02775... becoming cyborg: Activist Filmmaker, the Living Camera, Participatory Democracy, and Their Weaving "Throughout this article I use lowercase letters to deemphasize the importance of the individualized human in cyborg connection." http://irqr.ucpress.edu/content/10/4/340 http://irqr.ucpress.edu/content/10/4/340 Disaster Capitalism and the Quick, Quick, Slow Unravelling of Animal Life "Sea otters have barely survived centuries of colonial and capitalist development." https://onlinelibrary.wiley.com/doi/abs/10.1111/anti.12389 https://onlinelibrary.wiley.com/doi/abs/10.1111/anti.12389 Queer organising and performativity: Towards a norm-critical conceptualisation of organisational intersectionality http://www.ephemerajournal.org/sites/default/files/pdfs/issue/18-1ephemera-mar18.pdf#page=106 http://www.ephemerajournal.org/sites/default/files/pdfs/issu... Speciesism Party: A Vegan Critique of Sausage Party "In this article, we have described how Sausage Party reflects and reproduces intersecting oppressive power relations of species, gender, sexuality, ethnicity and different forms of embodiment." https://academic.oup.com/isle/advance-article/doi/10.1093/isle/isx075/4795356 https://academic.oup.com/isle/advance-article/doi/10.1093/is... "Wow, that bitch is crazy!" PhD Thesis about watching "Bachelor" with friends. https://search.proquest.com/openview/f4a6dbda2dc523609ad56c525553ea70/1?pq-origsite=gscholar&cbl=18750&diss=y https://search.proquest.com/openview/f4a6dbda2dc523609ad56c5... Diaries, dicks, and desire: how the leaky traveler troubles dominant discourse in the eroticized Caribbean Paper on being a sex tourist in the Caribbean. http://www.tandfonline.com/doi/full/10.1080/14766825.2011.653359 http://www.tandfonline.com/doi/full/10.1080/14766825.2011.65...
- knolan 9y agoCitation count isn’t always the best indicator depending on the field. It’s hard enough to find good papers without the literature being further polluted with substandard work. The job of a reviewer isn’t a simple yes/no answer. Reviewers often suggest sweeping changes to work to make it publishable. This helps to ensure that the literature is populated with well put together work that is free of bias and glaring mistakes. Otherwise we’d spend every minute helping students to spot the difference and wading through misleading research.
- GarrisonPrime 9y ago>Otherwise we’d spend every minute helping students to spot the difference and wading through misleading research. I already spend far too much time doing this. I can't imagine how futile literature research would be without peer review. There are already enough illogical conclusions and poor study designs to wade through.
- Someone 9y agoNumber of citations is gamed too easily. You would first get self-citations and, after you started ignoring those, citation rings (groups of people colluding to citate each other excessively). So, yes, you would need to assign a reputation to citations. I don’t think that’s a solved problem or even simpler than the problem of assigning reputation to papers.
- rocqua 9y agoThis is the same problem google faced and 'solved' by with the page-rank algorithm. I've seen some attempts at applying this to the reference graph, but it doesn't seem to be taking off.
- cornholio 9y agoIt's not taking off because, out of the box, it can't replace the gatekeepers: there needs to be a certain time period, sometimes decades, from publication until the moment a major work is recognized and cited. Cites form a DAG (old papers never cite newer ones) so do not converge on a recursive application of PageRank, newest works have all identical weights by definition. So in the short run, you still need a prospective measure of scientific value, and acceptance for publication in a selective journal ("impact factor") serves that purpose well.
- rocqua 9y agoAh of-course. I forgot the convergence requirements for pagerank. Indeed, the fact that cites are a DAG really does kind of screw with pagerank. Note that my supposed idea was to use 'pagerank' to replace reference count as the value of a work. Not to replace peer review.
- cup-of-tea 9y agoGood heavens no. A large amount of citations happen when somebody looks for something to cite and cites the first one that looks relevant.
- jclos 9y agoIf you're really interested in the question, there is a field of research called scientometrics and that is dedicated, among other things, to study this kind of question.
- anjc 9y agoThe key thing not being pointed out is that these papers are generally printed in a journal or as proceedings to a conference. So there needs to be a hard deadline by which the work needs to be complete, and seen as valid, for printing. So in that sense, there needs to be a peer review, because there aren't any take-backs at this stage.