3 ms·

"Publish -> people find insight and try to pick it apart -> You either have flaws or you get reproduced... Only then should your paper be of any worth to be quo

"Publish -> people find insight and try to pick it apart -> You either have flaws or you get reproduced... Only then should your paper be of any worth to be quoted or sighted from." This is already how it's supposed to work. The review before publication is a fairly superficial check that just confirms that what you describe follows basic scientific practices. There is no validation of the actual research. A proper reproduction is what's supposed to come after publication. IMO the real problems are that a) there isn't much glamour and funding for reproducing other's studies and b) "science journalists", university PR departments and now in part people on social media are picking up research before people on the field looked at it or misrepresent it. Suddenly the audience is a lot of folks who never were the intended audience of the process.

zer00eyz 3y ago
There is a pretty simple way to change all of that. Academic standards: You are not longer allowed to site a non reproduced paper in yours. Citations matter as much as the print, put the hurdle there and all of a sudden things will change real quick.
- freedomben 3y ago
  > You are not longer allowed to site a non reproduced paper in yours. I fully agree that in an ideal world, that would be the case. But some reproductions (especially now with machine learning) could cost millions of dollars and years to do. I don't think that's a reasonable or feasible thing to require.
  - BlueTemplar 3y ago
    Well, then, at least there should be pressure around having to mention this : a non-replicated study (by an independent group) is after all inherently suspect.
  - zer00eyz 3y ago
    >> could cost millions of dollars and years to do https://en.wikipedia.org/wiki/List_of_colleges_and_universities_in_the_United_States_by_endowment https://en.wikipedia.org/wiki/List_of_colleges_and_universit... Not only do they have the money they are charging the students more than ever.
    - freedomben 3y ago
      This is a very fair criticism and something that absolutely should be discussed, although I think it's a separate issue as papers can be written and published by anybody, regardless of backing from academic institutions.
- ramblenode 3y ago
  > Academic standards: You are not longer allowed to site a non reproduced paper in yours. > Citations matter as much as the print, put the hurdle there and all of a sudden things will change real quick. The reproducibility crisis is just a symptom of the publish-or-perish culture that pushes academics to churning out bad research. Academia already over-emphasizes publishing positive results at the expense of studying important questions. Your solution would further incentivize low risk, low impact research that we have too much of. Aside from that, there are a lot of edge cases that would make this difficult. If I do five studies that are modifications of each other, and all show the same basic effect, but I publish it as one paper, does that count as being reproduced? What if a result has only ever been reproduced within a single research group? Does the Higgs Boson need to be confirmed at at a collider outside the LHC?
  - zer00eyz 3y ago
    >> What if a result has only ever been reproduced within a single research group? Does the Higgs Boson need to be confirmed at at a collider outside the LHC? Yes. It does. Take out all the detectors and let another team build their own and come in and prove it. The really expensive part was the big concrete doughnut in the ground. After that wee could ... you know... shut the fuckin thing down so we stop pouring money into a literal hole in the ground. Idk we could do some more science after that. What is missing from quantum theory that the LHC running is going to find? IF we stopped funding mastrubatory string theory and put more of that into practical physics maybe we would have had something else for the LHC to do... >>> Academia already over-emphasizes publishing positive results at the expense of studying important questions Academia is made up of Academics, they aren't inclined to fix their own problem... it as effective as the church policing its own issues. I love what Jack Horner has done (see: https://creation.com/dino-puberty-blues https://creation.com/dino-puberty-blues ) but he would never have been able to achieve any of that if those people were alive and his peers.

zer00eyz 3y ago

There is a pretty simple way to change all of that. Academic standards: You are not longer allowed to site a non reproduced paper in yours. Citations matter as much as the print, put the hurdle there and all of a sudden things will change real quick.

freedomben 3y ago
> You are not longer allowed to site a non reproduced paper in yours. I fully agree that in an ideal world, that would be the case. But some reproductions (especially now with machine learning) could cost millions of dollars and years to do. I don't think that's a reasonable or feasible thing to require.
- BlueTemplar 3y ago
  Well, then, at least there should be pressure around having to mention this : a non-replicated study (by an independent group) is after all inherently suspect.
- zer00eyz 3y ago
  >> could cost millions of dollars and years to do https://en.wikipedia.org/wiki/List_of_colleges_and_universities_in_the_United_States_by_endowment https://en.wikipedia.org/wiki/List_of_colleges_and_universit... Not only do they have the money they are charging the students more than ever.
  - freedomben 3y ago
    This is a very fair criticism and something that absolutely should be discussed, although I think it's a separate issue as papers can be written and published by anybody, regardless of backing from academic institutions.
ramblenode 3y ago
> Academic standards: You are not longer allowed to site a non reproduced paper in yours. > Citations matter as much as the print, put the hurdle there and all of a sudden things will change real quick. The reproducibility crisis is just a symptom of the publish-or-perish culture that pushes academics to churning out bad research. Academia already over-emphasizes publishing positive results at the expense of studying important questions. Your solution would further incentivize low risk, low impact research that we have too much of. Aside from that, there are a lot of edge cases that would make this difficult. If I do five studies that are modifications of each other, and all show the same basic effect, but I publish it as one paper, does that count as being reproduced? What if a result has only ever been reproduced within a single research group? Does the Higgs Boson need to be confirmed at at a collider outside the LHC?
- zer00eyz 3y ago
  >> What if a result has only ever been reproduced within a single research group? Does the Higgs Boson need to be confirmed at at a collider outside the LHC? Yes. It does. Take out all the detectors and let another team build their own and come in and prove it. The really expensive part was the big concrete doughnut in the ground. After that wee could ... you know... shut the fuckin thing down so we stop pouring money into a literal hole in the ground. Idk we could do some more science after that. What is missing from quantum theory that the LHC running is going to find? IF we stopped funding mastrubatory string theory and put more of that into practical physics maybe we would have had something else for the LHC to do... >>> Academia already over-emphasizes publishing positive results at the expense of studying important questions Academia is made up of Academics, they aren't inclined to fix their own problem... it as effective as the church policing its own issues. I love what Jack Horner has done (see: https://creation.com/dino-puberty-blues https://creation.com/dino-puberty-blues ) but he would never have been able to achieve any of that if those people were alive and his peers.