4 ms·

This article points to a problem, but doesn't point to a solution.First of all, pre-approving papers isn't any significant improvement, since it really doesn't

by bagosm 13y ago

This article points to a problem, but doesn't point to a solution.First of all, pre-approving papers isn't any significant improvement, since it really doesn't remove the incentive to "cook" results, and besides if the journals knew how to filter papers based on the proposition and the setup, it wouldn't be any different if those papers were also accompanied by results. Putting this in a different wording, the journals can ignore the results and focus on the proposition and method if they know how to evaluate them. Also, judging a peer by his "quality" and not "quantity" of work is the only actual proposition, but it helps very little. If such objective criteria were known, journals would apply those criteria to the papers themselves, hence, filter only the quality ones and problem solved. All in all, the conclusion is that problem is solved iff problem is solved. What lacks, are objective criteria for a work of science. Anyone care to invent any?

Someone 13y ago
"If such objective criteria were known, journals would apply those criteria to the papers themselves, hence, filter only the quality ones and problem solved." If only. Just like currently, there would be second-tier of journals that will accept papers that are not the best of the best because all those papers make it into the top journals (Nature, Science, the Lancet, etc) Similarly, there would be third-tier journals gearing towards third-tier researchers, ad infinitum. People would send papers to them because their employer tells them "publish or perish", and a half/quarter/tenth-decent reputation is better than no reputation. Similarly, publishers would offer such journals because there is money to be made there. So yes, we need objective criteria for quality, but that will probably not be enough to get rid of the zillions of journals. Maybe, the real problem is that modern scientists are paid by the government. 'Scientist' used to be a hobby. People did it in their spare time, or found a patron who was willing to sponsor them. Nowadays, scientist is a profession, and the main sponsors (governments and governmental institutions) try to be fair in the way they distribute their money. To do so, they have to make scientific output measurable. Quality often can only be objectively measured after decades, so they go for quantity instead. The patrons of centuries ago did not have to defend their choices; they just sponsored whoever they wanted to.
- seanmcdirmid 13y ago
  You mean 'scientist' used to be a hobby for the otherwise well heeled and financially free gentry. Or if you could find a patron, but even then that was poor aristocrat being sponsored by richer aristocrat. Now that the plebes actually want to perform science for money...it has become so corrupted?
  - Someone 13y ago
    Corrupted in the sense "deviated from an ideal"? I am not sure, because I don't think patronage is the ideal. On the one hand, a patron Can answer "why does she get money, while I don't?" with "because". For a government, at the least the answer has to be "because she's better", which triggers "how do you know?". That, I think, is the root of "publish or perish". Now, to keep a patron, one might have to publish or perish, too. It's not like Mozart could spend a couple of years thinking of the great Austrian piano concerto without performing, composing stuff for various festivities, etc. However, the back side of this is that we lost many great minds. There just aren't that many patrons around.
    - seanmcdirmid 13y ago
      I think you are looking at history through rosy tinted glass. The patron system worked, but not as well as the government funded research we have today. It would never have gotten us to the moon.
      - Someone 13y ago
        Rosy tinted? I said "because I don't think patronage is the ideal", and only argued that it was good at preventing "publish or perish" for the third, fourth, etc. tier scientists. Thinking more about it, that may have been because it didn't give those scientists the "publish" option.
- jpdoctor 13y ago
  > and the main sponsors (governments and governmental institutions) try to be fair in the way they distribute their money. To do so, they have to make scientific output measurable. Your first sentence is right, but the second is off-base. No one in gov't really gives a damn about measurable progress, which is why the measurements used are so broken. (See my other comment for a better measure.) The real issue: As with any gov't cheese it needs to be spread around, or it doesn't get the congressional support required for continuation. Ever wonder why: 1. The National Magnet Lab was removed from MIT and placed in that bastion of great science, Florida State University? 2. The Superconducting Super Collider ended up in that great cultural center of science, Waxahachie, Texas? Perhaps ease of access? 3. Plans for Mission Control, which were already drawn up to be north of MIT, was redrawn to be in Houston after Kennedy was killed? (look up the history of Tech Square in Cambridge.) Politicians think they can spend money in certain areas and results will materials. And they will get results, so it's self-reinforcing, but they're too scientifically obtuse to imagine that the results are not necessarily the ones they want.

by bagosm 13y ago

Someone 13y ago

"If such objective criteria were known, journals would apply those criteria to the papers themselves, hence, filter only the quality ones and problem solved." If only. Just like currently, there would be second-tier of journals that will accept papers that are not the best of the best because all those papers make it into the top journals (Nature, Science, the Lancet, etc) Similarly, there would be third-tier journals gearing towards third-tier researchers, ad infinitum. People would send papers to them because their employer tells them "publish or perish", and a half/quarter/tenth-decent reputation is better than no reputation. Similarly, publishers would offer such journals because there is money to be made there. So yes, we need objective criteria for quality, but that will probably not be enough to get rid of the zillions of journals. Maybe, the real problem is that modern scientists are paid by the government. 'Scientist' used to be a hobby. People did it in their spare time, or found a patron who was willing to sponsor them. Nowadays, scientist is a profession, and the main sponsors (governments and governmental institutions) try to be fair in the way they distribute their money. To do so, they have to make scientific output measurable. Quality often can only be objectively measured after decades, so they go for quantity instead. The patrons of centuries ago did not have to defend their choices; they just sponsored whoever they wanted to.

seanmcdirmid 13y ago
You mean 'scientist' used to be a hobby for the otherwise well heeled and financially free gentry. Or if you could find a patron, but even then that was poor aristocrat being sponsored by richer aristocrat. Now that the plebes actually want to perform science for money...it has become so corrupted?
- Someone 13y ago
  Corrupted in the sense "deviated from an ideal"? I am not sure, because I don't think patronage is the ideal. On the one hand, a patron Can answer "why does she get money, while I don't?" with "because". For a government, at the least the answer has to be "because she's better", which triggers "how do you know?". That, I think, is the root of "publish or perish". Now, to keep a patron, one might have to publish or perish, too. It's not like Mozart could spend a couple of years thinking of the great Austrian piano concerto without performing, composing stuff for various festivities, etc. However, the back side of this is that we lost many great minds. There just aren't that many patrons around.
  - seanmcdirmid 13y ago
    I think you are looking at history through rosy tinted glass. The patron system worked, but not as well as the government funded research we have today. It would never have gotten us to the moon.
    - Someone 13y ago
      Rosy tinted? I said "because I don't think patronage is the ideal", and only argued that it was good at preventing "publish or perish" for the third, fourth, etc. tier scientists. Thinking more about it, that may have been because it didn't give those scientists the "publish" option.
jpdoctor 13y ago
> and the main sponsors (governments and governmental institutions) try to be fair in the way they distribute their money. To do so, they have to make scientific output measurable. Your first sentence is right, but the second is off-base. No one in gov't really gives a damn about measurable progress, which is why the measurements used are so broken. (See my other comment for a better measure.) The real issue: As with any gov't cheese it needs to be spread around, or it doesn't get the congressional support required for continuation. Ever wonder why: 1. The National Magnet Lab was removed from MIT and placed in that bastion of great science, Florida State University? 2. The Superconducting Super Collider ended up in that great cultural center of science, Waxahachie, Texas? Perhaps ease of access? 3. Plans for Mission Control, which were already drawn up to be north of MIT, was redrawn to be in Houston after Kennedy was killed? (look up the history of Tech Square in Cambridge.) Politicians think they can spend money in certain areas and results will materials. And they will get results, so it's self-reinforcing, but they're too scientifically obtuse to imagine that the results are not necessarily the ones they want.