3 ms·

You are right about the incentive structure being a main problem in science. But at the heart of the reasons why this structure is the way it is, you find peop

You are right about the incentive structure being a main problem in science. But at the heart of the reasons why this structure is the way it is, you find people's utter lack of knowledge about how science does and should work in the first place. Remarably, nearly all the commenters here betray this very lack of knowledge by talking about it as "philosophy". As if it was some practically irrelevant jibberjabber. Science does not work by magic though, with people going through some motions of an occult ritual. It's an algorithm, implemented insanely badly currently.

staunton 4y ago
> at the heart of the reasons why this structure is the way it is, you find people's utter lack of knowledge about how science does and should work I don't think people's lack of understanding scientific methods is the main factor shaping the incentive structure, though it's very interesting how big a role it plays and I would love to hear more detailed arguments. In order to change the system, there must be enough alignment on what the actual goals are and what incentives might benefit them, ways to effectively organize around it, and ways to actually enforce the new incentives. (Perhaps even more things...) I think there are big hurdles on each of the above. First, concerning alignment on the goal, many people don't agree that faster scientific progress is actually good. You might argue that trying to teach people epistemology might change that. I neither think it would nor that it should, but one could argue about it a lot. "Indoctrinating" people starting at a young age that "science and knowledge = good" as a moral value might work (it is being done and I mostly endorse that) but that's different from teaching the scientific method. The really important important factor here is, of course, that many individuals and organizations benefit from the current system and don't want to change it. Concerning organization, political intersts and competition hinder it somewhat, but co-operation between scientists is quite high and increasing. Hopefully that trend will continue and not sucumb to geopolitical pressures. Concerning enforcing better incentives, that's very hard. For example, we can't objectively and reliably evaluate someone's scientific contributions. Any metric you set up is going to be gamed, etc. I think the incentives have improved over the (very short) history of collaborative science and the trend looks positive. Take for example the trend towards open access publication and growing awareness of many issues (replication crisis, etc.). Generally, I would say that understanding how science (the system) works and how it should work (or would, given certain changes) is very hard. Similarly, I also think really deep understanding how science (the process) should work is also hard and applying that knowledge in practice is much harder still. At some point you get diminishing returns for the efforts of teaching it. You still haven't proposed how you would promote this knowledge. I can't think of anything that seems likely to do much good but would love to hear about it. > Remarably, nearly all the commenters here betray this very lack of knowledge by talking about it as "philosophy" I believe epistemology and philosophy of science are philosophy. I also believe that having and improving these things is very useful and one cannot overstate their role in getting science where it is now. This is in my mind perfectly consistent with there not being a great need (way?) to teach more of it to students. Perhaps an explanation of how you would teach it would make me realize we haven't been talking about quite the same thing?
- caddemon 4y ago
  It's harder to game metrics when there is a wider diversity of them and the way in which they are weighted in different decision making committees is different. Because it's hard to simultaneously game a bunch of different metrics, and the benefits from gaming a single one are probably worse than the benefits of just organically doing what you think is best. But yeah I totally agree that these are extremely hard problems with no silver bullet. Philosophically and from a "how do systems of humans work together" perspective. Teaching methodology of science like a cookbook I also agree is the wrong play and just digging in further to what got us where we are. I'm curious what makes you say scientists are collaborating more? Stats or has that been your experience? I've seen a lot of semi political collaborations where people are mostly just posturing to get on more papers. I have also seen a decent number of good faith interdisciplinary collaborations, though it seems to be a bit of a coin flip whether they flame out over disagreements. I've personally not seen much cooperation within the same field though that I'd consider genuine. Thanks for sparking the interesting discussion in any event, it's not often I end up debating people on both sides of a debate so this has been fun.
  - staunton 4y ago
    In my field there's lots of honest and fruitful collaboration. It's not all perfect but definitely a good situation. Technology has been making collaboration easier and faster as well. Globalization has led to international scientific collaboration being ever more expected and normal. Look into the organization of CERN for example. It's a huge project, highly politicized in many ways, with many parties that have their own goals and motives. Yet they all realize that a huge and sufficiently effective collaboration is the only way to do this science at all. And they are doing it.
caddemon 4y ago
I'm not sure why you think "philosophy" and "practically irrelevant jibberjabber" are synonymous. Or maybe you just think that's what commenters are implying? That's certainly not what I was trying to imply. Scientific thinking can be super practical, and it need not involve meta thinking for everyone that uses it. But there absolutely are philosophical discussions that remain of relevance today, discussions that some part of the scientific community ought to be having. Besides, academic science has become more formalized than it used to be, not less. People are running the algorithm badly IMO not because they are trying to be philosophical or eschew the previous fundamentals of science, but because they lack a deep understanding of what they are attempting to implement. The guardrails that were put up to try and force things to be more formalized have backfired into becoming a bad crutch for shitty science. It's also not surprising that certain metrics and incentive structures would break in a career environment where resources have become scarce. Things implemented in good faith 25 years ago may not have even been a bad idea at the time, but they definitely were reacting to a different era with different job expectations and different open questions. Anyway, I definitely agree with most of your broader concerns but I'm not sure if I'm misinterpreting your statement about philosophy. In my experience present grad students are hyperfixated on highly unimportant (from a training perspective) technical details. When I've been around PhD students chatting about science at lunch or house parties or whatever I more often hear discussions at the level of abstraction of mouse perfusion techniques than philosophy of science. It is certainly theoretically possible to have too many eggheads, but that's been the opposite of my experience.

staunton 4y ago

> at the heart of the reasons why this structure is the way it is, you find people's utter lack of knowledge about how science does and should work I don't think people's lack of understanding scientific methods is the main factor shaping the incentive structure, though it's very interesting how big a role it plays and I would love to hear more detailed arguments. In order to change the system, there must be enough alignment on what the actual goals are and what incentives might benefit them, ways to effectively organize around it, and ways to actually enforce the new incentives. (Perhaps even more things...) I think there are big hurdles on each of the above. First, concerning alignment on the goal, many people don't agree that faster scientific progress is actually good. You might argue that trying to teach people epistemology might change that. I neither think it would nor that it should, but one could argue about it a lot. "Indoctrinating" people starting at a young age that "science and knowledge = good" as a moral value might work (it is being done and I mostly endorse that) but that's different from teaching the scientific method. The really important important factor here is, of course, that many individuals and organizations benefit from the current system and don't want to change it. Concerning organization, political intersts and competition hinder it somewhat, but co-operation between scientists is quite high and increasing. Hopefully that trend will continue and not sucumb to geopolitical pressures. Concerning enforcing better incentives, that's very hard. For example, we can't objectively and reliably evaluate someone's scientific contributions. Any metric you set up is going to be gamed, etc. I think the incentives have improved over the (very short) history of collaborative science and the trend looks positive. Take for example the trend towards open access publication and growing awareness of many issues (replication crisis, etc.). Generally, I would say that understanding how science (the system) works and how it should work (or would, given certain changes) is very hard. Similarly, I also think really deep understanding how science (the process) should work is also hard and applying that knowledge in practice is much harder still. At some point you get diminishing returns for the efforts of teaching it. You still haven't proposed how you would promote this knowledge. I can't think of anything that seems likely to do much good but would love to hear about it. > Remarably, nearly all the commenters here betray this very lack of knowledge by talking about it as "philosophy" I believe epistemology and philosophy of science are philosophy. I also believe that having and improving these things is very useful and one cannot overstate their role in getting science where it is now. This is in my mind perfectly consistent with there not being a great need (way?) to teach more of it to students. Perhaps an explanation of how you would teach it would make me realize we haven't been talking about quite the same thing?

caddemon 4y ago
It's harder to game metrics when there is a wider diversity of them and the way in which they are weighted in different decision making committees is different. Because it's hard to simultaneously game a bunch of different metrics, and the benefits from gaming a single one are probably worse than the benefits of just organically doing what you think is best. But yeah I totally agree that these are extremely hard problems with no silver bullet. Philosophically and from a "how do systems of humans work together" perspective. Teaching methodology of science like a cookbook I also agree is the wrong play and just digging in further to what got us where we are. I'm curious what makes you say scientists are collaborating more? Stats or has that been your experience? I've seen a lot of semi political collaborations where people are mostly just posturing to get on more papers. I have also seen a decent number of good faith interdisciplinary collaborations, though it seems to be a bit of a coin flip whether they flame out over disagreements. I've personally not seen much cooperation within the same field though that I'd consider genuine. Thanks for sparking the interesting discussion in any event, it's not often I end up debating people on both sides of a debate so this has been fun.
- staunton 4y ago
  In my field there's lots of honest and fruitful collaboration. It's not all perfect but definitely a good situation. Technology has been making collaboration easier and faster as well. Globalization has led to international scientific collaboration being ever more expected and normal. Look into the organization of CERN for example. It's a huge project, highly politicized in many ways, with many parties that have their own goals and motives. Yet they all realize that a huge and sufficiently effective collaboration is the only way to do this science at all. And they are doing it.

caddemon 4y ago

I'm not sure why you think "philosophy" and "practically irrelevant jibberjabber" are synonymous. Or maybe you just think that's what commenters are implying? That's certainly not what I was trying to imply. Scientific thinking can be super practical, and it need not involve meta thinking for everyone that uses it. But there absolutely are philosophical discussions that remain of relevance today, discussions that some part of the scientific community ought to be having. Besides, academic science has become more formalized than it used to be, not less. People are running the algorithm badly IMO not because they are trying to be philosophical or eschew the previous fundamentals of science, but because they lack a deep understanding of what they are attempting to implement. The guardrails that were put up to try and force things to be more formalized have backfired into becoming a bad crutch for shitty science. It's also not surprising that certain metrics and incentive structures would break in a career environment where resources have become scarce. Things implemented in good faith 25 years ago may not have even been a bad idea at the time, but they definitely were reacting to a different era with different job expectations and different open questions. Anyway, I definitely agree with most of your broader concerns but I'm not sure if I'm misinterpreting your statement about philosophy. In my experience present grad students are hyperfixated on highly unimportant (from a training perspective) technical details. When I've been around PhD students chatting about science at lunch or house parties or whatever I more often hear discussions at the level of abstraction of mouse perfusion techniques than philosophy of science. It is certainly theoretically possible to have too many eggheads, but that's been the opposite of my experience.