5 ms·
I would say it is less that poor methods are rewarded, and more that the investment in proper methods is expensive, and thus penalized. That is, nobody is part
by taeric 10y ago
I would say it is less that poor methods are rewarded, and more that the investment in proper methods is expensive, and thus penalized.
That is, nobody is particular looking for people doing poor work and handing out rewards. However, the "proper" methodology that we want necessitates time. Something that is expensive and there are already plenty of things eating at the budgets of work out there.
I do think this can be made better. But I see no reason to think it is just us chiding people for not doing better.
- btilly 10y agoIt makes no difference whether you portray it as "poor methods are rewarded" or "good methods are penalized", either way the incentives are for poor methods to win out over good ones.
- taeric 10y agoIt is actually a huge difference. Raising kids, for instance, it isn't enough to just reward good behavior. You have to actively and fairly penalize bad behavior. And if you are setting up a losing game or your kids, expect bad behavior. Which is what most of this comes down to. For most participants, they are in a losing game. The absolute best they can do is get through with their project and move to the next one. About the worst they can do is spend more time on what they are currently doing. So, I don't want my comment to be a "screw this article, they got it wrong." No, contrarily, this is a discussion that needs to be had. I personally think it is not enough to just look at the rewards people set themselves up for on the bad behavior. You have to look at the penalties they have for "good" behavior. This is not new. Few years ago the lament was we don't teach how to fail. That is all I am echoing. Failure is heavily punished. Maybe it shouldn't be.
- btilly 10y agoThe article found that even strong punishments for failure were not enough in their toy model. So it isn't as simple as just punishing discovered bad behavior. I'm reminded of an economics article that I read on cheating in sports. It turns out that no amount of punishment of cheaters would be enough to stop cheating in bicycling. What would be required is that if ANY bicyclist is caught cheating, EVERYONE who had worked with him who hadn't reported it would have to be punished. Don't just punish the cheaters, punish those in a position to have conceivably known who didn't report it. The scientific equivalent would be to not punish bad research articles, but to punish everyone who went through the lab who could have raised the flag and didn't. I'm pretty sure that they didn't try modeling that. It would have been interesting to know what that would have shown in their model.
- taeric 10y agoMy question is how do you model the ridiculous punishment that they already face for not getting done with their work? In the sports world, there is already a huge punishment for not winning. To the point that most punishments of cheating are actually just on par with never having competed at that level in the first place. That is, don't just try to dream up punishments for people that act in negative ways. Try to come up with scenarios that don't require superhuman levels of achievement just to play. Make sure failure is OK at all levels. As it is, many look at the punishment of getting caught acting negatively, and the reality of failing, and conclude somewhat rationally, that cheating might be worth it.
- tpeo 10y agoI don't think such a system would work out, or at least it would breed a huge amount of resentment towards it. Vaguely reminds me of Legalist philosophy: it attempted to "abolish penalties by the means of penalties", that is, through a liberal use of (capital) punishment not only for criminals but for their whole families, so that no one would dare commit a crime. https://en.wikipedia.org/wiki/Legalism_%28Chinese_philosophy%29 https://en.wikipedia.org/wiki/Legalism_%28Chinese_philosophy...
- manarth 10y agoDon't just punish the cheaters, punish those in a position to have *conceivably known* who didn't report it. There's a significant difference between punishing those who did know about it (and didn't report it) and those who might have known about it (and didn't report it). The second sounds a lot like "collective punishment", which is prohibited by the Fourth Geneva Convention, and the Human Rights Act. [1] [1] https://ihl-databases.icrc.org/customary-ihl/eng/docs/v1_rul_rule103 https://ihl-databases.icrc.org/customary-ihl/eng/docs/v1_rul...
- posterboy 10y ago> the penalties they have for "good" behavior. Do you mean positive punishment? > You have to actively and fairly penalize bad behavior > Failure is heavily punished. Maybe it shouldn't be. Fair is the keyword, but I didn't understand the part quoted last, until I read the VikingCoder's comment. Edit: and now there are sibling-comments regarding that point.
- taeric 10y agoProbably more accurate to say the risks for good behavior. Basically, playing fair and losing is a huge cost.
- rodgerd 10y agoI disagree completely. Ackowledging that "good methods are penalised" would challenge the line generally taken by the Economist in its socio-political pieces: that if classical liberal economic models were applied to everything, everything would work better. What's being described here is the consequence of the application of "business thinking" and market-driven funding of science, with all the unintended consequences of trying to apply quarterly metrics to pure research. Suggesting the problem is "not enough punishment" avoids having to ask philosophically hard questions.
- taeric 10y agoI believe we are taking past each other. I specifically do not want more punishment. I want cheaper failures. I want it such that if you spend ten years chasing a dead end, you aren't a failure. You can still have a successful and happy family where your kids have a fair shot, as well. That make sense?
- AstralStorm 10y agoThis would require a tax for science, as all kinds of corporation funding goes to promote bad behaviour such as requiring immediate results. Specifically, much higher level of public funding and more positions to fill.
- hannob 10y ago> That is, nobody is particular looking for people doing poor work and handing out rewards. I'd challenge that. If you look for people with high impact factors you are asking for people doing poor science. If you only publish "significant" results and refuse to publish negative results you are asking for bad science. There are structural problems that lead to bad science, and often it's not about costs.
- taeric 10y agoI argue that you are modeling out real expense. In this case, the cost of not publishing significant findings. You could try to penalize bad faith actors. You could also try to stop putting undue pressure on good faith actors. It isn't just spending money. But opportunity cost of throw away work. As it is we have what can be modeled akin to baseball where most at bats are failures, but not even swinging is frowned upon and eventually leads to you not being sent to bat.
- afarrell 10y agoGiven that plan to decimate the economy of America's 4th largest and most racially diverse city (Houston, TX), I am quite keen to know if these issues also affect climate science. Don't get me wrong, I'm still pretty convinced that anthropogenic global warming is happening. But if there is a change that it isn't and we can avoid all of the murders and suicides that come with throwing large numbers of people in one area out of work, we should investigate the possibility.
- lmm 10y agoThese issues affect all science. That's not an argument for ignoring science when it's politically convenient. (At least not any more than you already were based on the results. Whatever your all-in number for how often science is right or wrong is, that number already incorporates all these issues). If anything I would expect standards to be much higher in climate science than less controversial areas. There are a lot of people who would stand to gain (or, equivalently, avoid losing) huge amounts of money if the climate science was wrong, they will be going over papers with a fine-tooth comb in a way that doesn't happen to most papers.
- fma 10y agoNot only that but there's expectations of people finding positive results. https://en.wikipedia.org/wiki/Hwang_Woo-suk https://en.wikipedia.org/wiki/Hwang_Woo-suk Although Thomas Edison said "“I have not failed. I've just found 10,000 ways that won't work."...no grant writer wants to hear that.