3 ms·

The policies that cause 'publish or perish' to flourish are also heavily to blame here. If a scientist only is expected (or directly limited) to publish a hand

by chaxor 3y ago

The policies that cause 'publish or perish' to flourish are also heavily to blame here. If a scientist only is expected (or directly limited) to publish a handful of extremely good quality articles throughout their lifetime, they will likely make much more sure that the results are real.

chongli 3y ago
The policies that cause 'publish or perish' to flourish What policies are those? I thought 'publish or perish' was an expected outcome given the extreme supply/demand situation in the academic job market.
- barelysurviving 3y ago
  My gf is in academia (in economics) and she told me that to get a tenure you sign contracts demanding you’ll publish 4-5 relevant research papers in 5 years. A geologist friend of mine told me something similar, just with different numbers.
karaterobot 3y ago
I agree that the up-or-out culture of tenure is bad, and so are many other terrible university policies and practices. But I would not go so far as to ascribe blame to them. 100% of the responsibility goes to the people who knowingly falsified data to advance their careers. That doesn't mean the culture shouldn't change.
- godelski 3y ago
  > 100% of the responsibility goes to the people who knowingly falsified data to advance their careers. I don't think anyone would argue otherwise. The issue is that if there's a culture that encourages (incentivizes, not advocates) cheating, then we should expect a culture where more people cheat. I think right now it is hard to argue that the current culture doesn't encourage cheating, and if we're being honest, doesn't have a good means of identifying it (publish or perish also decreases reviewing quality).
moralestapia 3y ago
>The policies that cause 'publish or perish' to flourish are also heavily to blame here. Ugh. There's plenty of honest researchers in the field. You don't have to be corrupt to build a nice career and, now that I think of it, only people who are equally corrupt seem to embrace/justify/tolerate these practices.
godelski 3y ago
These policies also just make for bad science in general. In science you need people challenging the status quo, not everyone, but a significant percentage. A publish or perish paradigm encourages railroading in intellectual thought as well as encourages laziness in evaluation, as there is increased pressure on all researchers who are also the ones performing reviews. People who have zero incentive to perform a good review and many incentives to do a quick review (which is even actively encourage). This results in an over-reliance on benchmarkism, which we see in many fields. In ML we see SOTA chasing, forgetting that datasets are proxies which have their own biases and limitations as well as the same for our evaluation methods. In many of the softer sciences -- such as psychology or medicine -- we see a reliance on p-values, forgetting that 0.05 is an arbitrary value and not meaningful without additional context. All our sciences are converging on Goodhart's "Warning." There is utility in both speed and in a slow step. The truth is that we need both. A continuum of methods and ideas. After all, we are searching in the dark and our only real advantage is parallel Monte Carlo search. The problem is that this is incredibly difficult to accurately evaluate and we are arrogantly attempting to evaluate exceptionally noisy systems with extreme precision without even having a basic model of that system. We could relate this to businesses too, but the timetables for them are typically shorter and so the chaff can more clearly be gained (often post-explained through pure intent instead of it's combination with luck). Then again, we are trying to make academia a business, trying to apply their noisy successes to a completely different model.
ChainOfFools 3y ago
They're also expected, in a research university, to raise a constant stream of grant money. a certain subset of academics are starting to figure out that this becomes much easier when you invest a comparative pittance in a decent wardrobe , makeup , and good lighting so as to cast your research net extremely wide into populous rhetoric, Ted talks etc. The effort to reward ratio is fantastic as it invariably results in being contacted after your show, and it is a show, by people who have access to public outreach funds but very little understanding of the field nor your place in it, but want to be attached to whatever momentum you have and we'll throw money at you. This is multiplied several-fold when you are doing this populist intellectual dinner talk circuit in one of several countries whose elite are anxious to establish (or regain) their own scientific prestige on the world academic stage and will throw unbelievable amounts of money at the most brittle and flimsy proposals so long as there is an R1 institution attached.

by chaxor 3y ago

chongli 3y ago

The policies that cause 'publish or perish' to flourish What policies are those? I thought 'publish or perish' was an expected outcome given the extreme supply/demand situation in the academic job market.

barelysurviving 3y ago
My gf is in academia (in economics) and she told me that to get a tenure you sign contracts demanding you’ll publish 4-5 relevant research papers in 5 years. A geologist friend of mine told me something similar, just with different numbers.

karaterobot 3y ago

I agree that the up-or-out culture of tenure is bad, and so are many other terrible university policies and practices. But I would not go so far as to ascribe blame to them. 100% of the responsibility goes to the people who knowingly falsified data to advance their careers. That doesn't mean the culture shouldn't change.

godelski 3y ago
> 100% of the responsibility goes to the people who knowingly falsified data to advance their careers. I don't think anyone would argue otherwise. The issue is that if there's a culture that encourages (incentivizes, not advocates) cheating, then we should expect a culture where more people cheat. I think right now it is hard to argue that the current culture doesn't encourage cheating, and if we're being honest, doesn't have a good means of identifying it (publish or perish also decreases reviewing quality).

moralestapia 3y ago

>The policies that cause 'publish or perish' to flourish are also heavily to blame here. Ugh. There's plenty of honest researchers in the field. You don't have to be corrupt to build a nice career and, now that I think of it, only people who are equally corrupt seem to embrace/justify/tolerate these practices.

godelski 3y ago

These policies also just make for bad science in general. In science you need people challenging the status quo, not everyone, but a significant percentage. A publish or perish paradigm encourages railroading in intellectual thought as well as encourages laziness in evaluation, as there is increased pressure on all researchers who are also the ones performing reviews. People who have zero incentive to perform a good review and many incentives to do a quick review (which is even actively encourage). This results in an over-reliance on benchmarkism, which we see in many fields. In ML we see SOTA chasing, forgetting that datasets are proxies which have their own biases and limitations as well as the same for our evaluation methods. In many of the softer sciences -- such as psychology or medicine -- we see a reliance on p-values, forgetting that 0.05 is an arbitrary value and not meaningful without additional context. All our sciences are converging on Goodhart's "Warning." There is utility in both speed and in a slow step. The truth is that we need both. A continuum of methods and ideas. After all, we are searching in the dark and our only real advantage is parallel Monte Carlo search. The problem is that this is incredibly difficult to accurately evaluate and we are arrogantly attempting to evaluate exceptionally noisy systems with extreme precision without even having a basic model of that system. We could relate this to businesses too, but the timetables for them are typically shorter and so the chaff can more clearly be gained (often post-explained through pure intent instead of it's combination with luck). Then again, we are trying to make academia a business, trying to apply their noisy successes to a completely different model.

ChainOfFools 3y ago

They're also expected, in a research university, to raise a constant stream of grant money. a certain subset of academics are starting to figure out that this becomes much easier when you invest a comparative pittance in a decent wardrobe , makeup , and good lighting so as to cast your research net extremely wide into populous rhetoric, Ted talks etc. The effort to reward ratio is fantastic as it invariably results in being contacted after your show, and it is a show, by people who have access to public outreach funds but very little understanding of the field nor your place in it, but want to be attached to whatever momentum you have and we'll throw money at you. This is multiplied several-fold when you are doing this populist intellectual dinner talk circuit in one of several countries whose elite are anxious to establish (or regain) their own scientific prestige on the world academic stage and will throw unbelievable amounts of money at the most brittle and flimsy proposals so long as there is an R1 institution attached.