4 ms·

gatekeeping in science is a theoretical good...the difficulty in practice is highlighted by the general 'three paths' for citations...articles that never get ci

by avs733 8y ago

gatekeeping in science is a theoretical good...the difficulty in practice is highlighted by the general 'three paths' for citations...articles that never get cited, articles that get cited a lot at first and then are forgotten, and articles that aren't cited until much later and then cited en masse. That last category is really the 'good science' because it does what we want foundational research to do - disrupt current and limiting ways of thinking. Dialogue is an unmitigated good, but dialogue also tends to be dominated by the normative voice. Where science is different than OSPs is that OSPs need to work with what exists now, whereas science needs to engage in tension and informed dialogue in a more foudnational way..

Nasrudith 8y ago
Here is a question what is the sort of theoretical good for the gatekeeping? What is the functional purpose of it? I think that answer would prove insightful for determining the future. I can see two noncynical reasons for it one. One is educational purposes. Two would be quality control - if it is already debunkable or widely known there is no point in publishing in Nature "thinking done in the brain". The cynical abuses and perverse incentives however are myraid and academic politicking is already infamous. I don't know what solution will work but understanding the system goals and functions should help decide what to replace and what to keep.
- avs733 8y ago
  >Here is a question what is the sort of theoretical good for the gatekeeping? What is the functional purpose of it? I think that answer would prove insightful for determining the future. Because I am not at all prepared to assess the quality or accuracy of a quantum physics paper and I doubt a molecular biologist is prepared to assess the quality or accuracy of a sociology paper. In what I have seen there are a lot of cargo cultists who think kthey understand a field, and use big words, and just don't get it. I feel that there should be some filtering function to remove that. As an example: This guy: https://www.mountainproject.com/forum/topic/113602967/near-miss-on-cathedral-peak https://www.mountainproject.com/forum/topic/113602967/near-m... This guy: https://www.reddit.com/r/badphilosophy/comments/7x3t1g/stand_the_test_of_time_where_did_t https://www.reddit.com/r/badphilosophy/comments/7x3t1g/stand... This guy: http://ecclesiastes911.net/ http://ecclesiastes911.net/ and This guy: https://arxiv.org/search/math?searchtype=author&query=Simkin%2C+M https://arxiv.org/search/math?searchtype=author&query=Simkin... are all the same person.
  - x220 8y ago
    That guy has some pretty interesting articles.
mcbits 8y ago
I'd favor something like clustering and client-side filtering over gatekeeping. Mainstream academic research can coexist just fine with fringe theories and industry research in the same database as long as one can efficiently distinguish between them. And keeping everything together in one (de)central database would make it possible for people, so inclined, to annotate other people's work with new references to support or debunk the work long after it was published, or to clarify ambiguous language, etc. Those annotations, too, could be subject to filtering as needed. People could build reputations and whole careers around tying up loose ends instead of the "publish or perish" grind.
- nl 8y ago
  Mainstream academic research can coexist just fine with fringe theories and industry research in the same database as long as one can efficiently distinguish between them How do you propose that is done? For my job I build neural networks for text processing. I spend a lot of time reading papers in the field. And yet if I look at something in an adjacent field (even something as close as something like open information extraction) I have trouble telling which papers are important. How on earth am I supposed to tell if something in a further removed field which attracts more crackpots (say probability theory or something) is a fringe theory or a breakthrough from a new author? I'd note the example of the Gaussian correlation inequality[1] where even people in the field weren't aware it had been proven for 3 years after publication[2]. [1] https://www.quantamagazine.org/statistician-proves-gaussian-correlation-inequality-20170328/ https://www.quantamagazine.org/statistician-proves-gaussian-... [2] https://en.wikipedia.org/wiki/Gaussian_correlation_inequality https://en.wikipedia.org/wiki/Gaussian_correlation_inequalit...
  - mcbits 8y ago
    Arxiv accidentally created a crackpot filter from apparently a simple semantic classifier along with a count of stop words used in the article, which acts as a very coarse "style" metric.[1] Reasonable-looking crackpot work is usually classified as "general physics" instead of being rejected entirely. Unfortunately it also lumps some legitimate but unconventional research in with the crackpots and makes Arxiv itself somewhat of a gatekeeper. What I'm imagining is a sort of layered approach with raw "article" (or some other unit) data at the bottom and indexing, tagging, clustering, filtering, reviewing, commenting, linking, etc. layered on top with possibly many implementations to choose from. The analysis/filtering would also apply to people augmenting the data, so if some users are really good at tagging certain types of junk as junk, you could easily filter out that junk. If there emerges a cluster of users who keep tagging certain interesting material as "woo" then you could filter them out or even use them to discover interesting material. I doubt there's a silver bullet (at least today) that could reliably distinguish between unconventional-bad and unconventional-good work, but keeping the baby and the bathwater together opens the door to such an algorithm in the future. [1] https://arxiv.org/abs/1603.03824 https://arxiv.org/abs/1603.03824

by avs733 8y ago

Nasrudith 8y ago

Here is a question what is the sort of theoretical good for the gatekeeping? What is the functional purpose of it? I think that answer would prove insightful for determining the future. I can see two noncynical reasons for it one. One is educational purposes. Two would be quality control - if it is already debunkable or widely known there is no point in publishing in Nature "thinking done in the brain". The cynical abuses and perverse incentives however are myraid and academic politicking is already infamous. I don't know what solution will work but understanding the system goals and functions should help decide what to replace and what to keep.

avs733 8y ago
>Here is a question what is the sort of theoretical good for the gatekeeping? What is the functional purpose of it? I think that answer would prove insightful for determining the future. Because I am not at all prepared to assess the quality or accuracy of a quantum physics paper and I doubt a molecular biologist is prepared to assess the quality or accuracy of a sociology paper. In what I have seen there are a lot of cargo cultists who think kthey understand a field, and use big words, and just don't get it. I feel that there should be some filtering function to remove that. As an example: This guy: https://www.mountainproject.com/forum/topic/113602967/near-miss-on-cathedral-peak https://www.mountainproject.com/forum/topic/113602967/near-m... This guy: https://www.reddit.com/r/badphilosophy/comments/7x3t1g/stand_the_test_of_time_where_did_t https://www.reddit.com/r/badphilosophy/comments/7x3t1g/stand... This guy: http://ecclesiastes911.net/ http://ecclesiastes911.net/ and This guy: https://arxiv.org/search/math?searchtype=author&query=Simkin%2C+M https://arxiv.org/search/math?searchtype=author&query=Simkin... are all the same person.
- x220 8y ago
  That guy has some pretty interesting articles.

mcbits 8y ago

I'd favor something like clustering and client-side filtering over gatekeeping. Mainstream academic research can coexist just fine with fringe theories and industry research in the same database as long as one can efficiently distinguish between them. And keeping everything together in one (de)central database would make it possible for people, so inclined, to annotate other people's work with new references to support or debunk the work long after it was published, or to clarify ambiguous language, etc. Those annotations, too, could be subject to filtering as needed. People could build reputations and whole careers around tying up loose ends instead of the "publish or perish" grind.

nl 8y ago
Mainstream academic research can coexist just fine with fringe theories and industry research in the same database as long as one can efficiently distinguish between them How do you propose that is done? For my job I build neural networks for text processing. I spend a lot of time reading papers in the field. And yet if I look at something in an adjacent field (even something as close as something like open information extraction) I have trouble telling which papers are important. How on earth am I supposed to tell if something in a further removed field which attracts more crackpots (say probability theory or something) is a fringe theory or a breakthrough from a new author? I'd note the example of the Gaussian correlation inequality[1] where even people in the field weren't aware it had been proven for 3 years after publication[2]. [1] https://www.quantamagazine.org/statistician-proves-gaussian-correlation-inequality-20170328/ https://www.quantamagazine.org/statistician-proves-gaussian-... [2] https://en.wikipedia.org/wiki/Gaussian_correlation_inequality https://en.wikipedia.org/wiki/Gaussian_correlation_inequalit...
- mcbits 8y ago
  Arxiv accidentally created a crackpot filter from apparently a simple semantic classifier along with a count of stop words used in the article, which acts as a very coarse "style" metric.[1] Reasonable-looking crackpot work is usually classified as "general physics" instead of being rejected entirely. Unfortunately it also lumps some legitimate but unconventional research in with the crackpots and makes Arxiv itself somewhat of a gatekeeper. What I'm imagining is a sort of layered approach with raw "article" (or some other unit) data at the bottom and indexing, tagging, clustering, filtering, reviewing, commenting, linking, etc. layered on top with possibly many implementations to choose from. The analysis/filtering would also apply to people augmenting the data, so if some users are really good at tagging certain types of junk as junk, you could easily filter out that junk. If there emerges a cluster of users who keep tagging certain interesting material as "woo" then you could filter them out or even use them to discover interesting material. I doubt there's a silver bullet (at least today) that could reliably distinguish between unconventional-bad and unconventional-good work, but keeping the baby and the bathwater together opens the door to such an algorithm in the future. [1] https://arxiv.org/abs/1603.03824 https://arxiv.org/abs/1603.03824