3 ms·

The majority of content in most journals is asinine or incorrect. About the only thing that can fix this is quicker turnaround (cf STAP, NgAgo, the godawful GWA

by apathy 10y ago

The majority of content in most journals is asinine or incorrect. About the only thing that can fix this is quicker turnaround (cf STAP, NgAgo, the godawful GWAS paper in Neuron recently, etc). It's not at all clear to me that the checks or balances were helping beforehand. Hiring a shit ton of MBA types to administer research probably didn't help much either. Also, out of curiosity, is this open access / fast review problem what led to machine learning falling into irrelevance? Because we wouldn't want that to happen to other fields. Can you imagine if raw clinical trial data was out there, for example? (The Horror! Best keep these appallingly expensive results locked up!)

frozenport 10y ago
>>fix this is quicker turnaround Why? As the first author of several publications, more than half the reviews I received were not well thought out. This part of the process doesn't happen, Open Acess and self publishing websites exploited this fact for greater throughput. >>machine learning I would argue that the machine learning bloom in recent years is because of an availability of new technology from industry, supported by industry. This is also why they were able to tolerate less conventional venues for publication - as these were not viewed as mile markers for progress. Yet, when I look at more "settled" machine learning fields like speech, I see the same plethora of academic papers filled with useless crap. Perhaps the growth of ML has nothing to do with Open Acess. There is although certainly there is an underlying profit motive for successful technologies, Google ain't going to use a technique that doesn't work. >>raw clinical trial data It would be great if it were published, and indeed many people would pay for quality trial data. I see OA and self-publishing of research results having no relation to authors withholding clinical data.
- apathy 10y ago
  Re: 1: no shit. The turnaround is for rebuttals. I try to do thorough reviews (I have published plenty as well) but realistically I recognize that no one has the sort of free cycles required to catch everything in advance. What's worse is that an awful lot of reviews are pure political bullshit. In the end it's the people who may or may not try to build on your work (or mine!) that are the best judges of whether it's BS. Re: 2: probably, but diffusion of ideas benefits massively from readily accessible documentation. One of the things that will cause me to immediately reject a paper is if its method implementation does not work as described. I will always reject in such cases. They are alarmingly common and this is one of the few easy cases in reviewing. Lack of an implementation is also cause to reject. Re: 3: it has everything to do with it. Unless you're running and publishing trials please don't lecture me on this. See opentrials for a particularly frightening take on why published trials so poorly represent those submitted to the FDA or registered with CTEP. Meanwhile actual RCTs get buried by shitbags demanding more subjects than there are recorded cases in the past 40 years (no joke). If you're not doing trials, you may have to take my word for it (or get involved in RCTs and find out for yourself). A lot of the enterprise is actively harmful to science by omission.

by apathy 10y ago

frozenport 10y ago

>>fix this is quicker turnaround Why? As the first author of several publications, more than half the reviews I received were not well thought out. This part of the process doesn't happen, Open Acess and self publishing websites exploited this fact for greater throughput. >>machine learning I would argue that the machine learning bloom in recent years is because of an availability of new technology from industry, supported by industry. This is also why they were able to tolerate less conventional venues for publication - as these were not viewed as mile markers for progress. Yet, when I look at more "settled" machine learning fields like speech, I see the same plethora of academic papers filled with useless crap. Perhaps the growth of ML has nothing to do with Open Acess. There is although certainly there is an underlying profit motive for successful technologies, Google ain't going to use a technique that doesn't work. >>raw clinical trial data It would be great if it were published, and indeed many people would pay for quality trial data. I see OA and self-publishing of research results having no relation to authors withholding clinical data.

apathy 10y ago
Re: 1: no shit. The turnaround is for rebuttals. I try to do thorough reviews (I have published plenty as well) but realistically I recognize that no one has the sort of free cycles required to catch everything in advance. What's worse is that an awful lot of reviews are pure political bullshit. In the end it's the people who may or may not try to build on your work (or mine!) that are the best judges of whether it's BS. Re: 2: probably, but diffusion of ideas benefits massively from readily accessible documentation. One of the things that will cause me to immediately reject a paper is if its method implementation does not work as described. I will always reject in such cases. They are alarmingly common and this is one of the few easy cases in reviewing. Lack of an implementation is also cause to reject. Re: 3: it has everything to do with it. Unless you're running and publishing trials please don't lecture me on this. See opentrials for a particularly frightening take on why published trials so poorly represent those submitted to the FDA or registered with CTEP. Meanwhile actual RCTs get buried by shitbags demanding more subjects than there are recorded cases in the past 40 years (no joke). If you're not doing trials, you may have to take my word for it (or get involved in RCTs and find out for yourself). A lot of the enterprise is actively harmful to science by omission.