5 ms·

I generally agree, but their solution (fund boring research, publish only in journals with high standard) is in direct contrast to what they stated earlier. It'

I generally agree, but their solution (fund boring research, publish only in journals with high standard) is in direct contrast to what they stated earlier. It's basically saying yes, we have bad incentives, but we could ignore them. That's not going to happen; shiny new research will attract people and funding. Especially more than "boring" what we found before was indeed a finding-research. Now, I don't have a good solution either, unfortunately. What might work is that we require replication work for a PhD or have a certain percentage of a journal dedicated to verification. That, combined with some meta-studies to reward people with citations for replication, might work without fully swimming against the current. It's a hard problem, really.

searchableguy 6y ago
A more radical solution: Basic income for all researchers and provide efficient digital way to see how funds are being spent like open collective for open source. Most problems arise at an individual level where people don't get paid unless they are putting research papers on the new hype thing and the other is expenditure on middlemen and bureaucracy which isn't easily visible. Eliminate the use of research grants for funding people's living. Bit pedantic but you should use they instead of he/she. People can have other pronouns.
- kebman 6y ago
  Oh, you mean tenure? UBI is a general thing for the whole population, and not something just for scientists. Tenure is in general a good, as long as scientists keep working, and there are some controls keeping them producing at least "something". And let's face it, most of them do. This is great, because it releaves scientists of pressure, which means more of them can be creative in their research, which actually speeds up discoveries. This is also a valid approach within tech and IT btw. But it should be said that it has a very poor track record outside of these knowledge-heavy professions. On the other hand, if you really think UBI is a good solution, then I'm afraid you don't know how an economy really works.
  - searchableguy 6y ago
    Yeah, I meant something like tenure but less restricted for anyone doing research. > On the other hand, if you really think UBI is a good solution, then I'm afraid you don't know how an economy really works. Could you elaborate more?
    - kebman 6y ago
      It has been discussed to death, but for the sake of argument, let's review these two completely opposing factions, who both agree that UBI doesn't work: Zero Hedge on why UBI doesn't work: https://safehaven.com/markets/economy/Why-Universal-Basic-Income-Wont-Work.html https://safehaven.com/markets/economy/Why-Universal-Basic-In... The Guardian on why UBI doesn't work: https://www.theguardian.com/commentisfree/2019/may/06/universal-basic-income-public-realm-poverty-inequality https://www.theguardian.com/commentisfree/2019/may/06/univer...
      - cousin_it 6y ago
        Your second link mentions "universal basic services", I always supported that idea but didn't know it had a name. It's better than UBI because it sidesteps the problem of cost disease (landlords raising rents in response, etc).
        kebman 6y ago
        Why not go for the original idea, and go straight for Communism instead?
  - mattkrause 6y ago
    I think they mean something like UBI: non- or less-competitive grants that would fund modest-sized projects (or pilot studies for larger ones). Tenure doesn't really help with this problem in the lab sciences. While it lets faculty members keep their jobs, it usually doesn't come with enough funding to do experimental work. In fact, while you can "keep" a tenured position without grants, many places find ways to...discourage that (move your office to a shoebox in the sub-basement, crappy teaching and service assignments).
- gen155954609803 6y ago
  >Bit pedantic but you should use they instead of he/she. People can have other pronouns. One of those other pronouns is literally "they", so you're still risking using the wrong pronoun by using "they" for someone who prefers to be referred to as "he". Was anyone else taught the rule that when writing about a third-party whose gender is unknown, use your own pronouns?
- 0d9eooo 6y ago
  IIRC, Hungary has implemented something like this and it's been floated as an idea, to distribute grant dollars as a fixed amount to everyone who has an H index or citation count above a certain amount. When I've talked about this with colleagues, they've argued this would make problems worse, and it might, but I think the idea was more that once you reach a certain threshold you should be funded some amount. Of course, I'd argue that this is basically the idea behind tenure at an R1 institution, that once you reach a certain level the state or private board of trustees is paying your salary to do research. But nowadays tenure at an R1 institution is proxy for external grant dollars, which defeats the purpose and is redundant. The real problem is indirect funds, which create profit for institutions. The federal government needs to eliminate indirect funds, so that grant dollars aren't perversely incentivised and tenure is based on research quality rather than profit.
asutekku 6y ago
On an unrelated note, just use theirs instead if his/her. It makes the text easier to read and is the grammatically correct way.
- Sebb767 6y ago
  Thanks for the suggestion! I've edited the question.
setgree 6y ago
Tyler Cowen writes [0] that the most important question in economics, to him, is > how do differences of culture — however defined — interact with traditional economic mechanisms involving prices, incomes, and simple comparative statics? Are those competing explanations, namely cultural vs. economic? Richie's answers are mostly focused on changing the culture of science, and while there are lots of ways we could change the incentives, none of them would be pretty. Example: let's say we want to more closely align research and actionable results, e.g., a product a company can use (Brian Armstrong argues for something like this [1]). Solution: radically reduce public funding for scientific research and for university education as a whole (in line with Bryan Caplan's arguments in "The Case Against Education" [2]). Academics, who would be many fewer in number, would then have to get more of their funding from companies, who (presumably) would: A) guide them towards asking market-relevant questions, and B) have a clear incentive to check the data, re-run the code, etc. -- so that the product they built based on that research didn't flop. I think most people would recoil at this proposal. But that's what comes to mind when I think about fixing the incentives rather than the culture. P.S. Small nitpick: Richie gives an _example_ of a perverse incentive in lieu of a definition. [0] https://marginalrevolution.com/marginalrevolution/2017/01/important-unsolved-problems-field.html https://marginalrevolution.com/marginalrevolution/2017/01/im... [1] https://medium.com/@barmstrong/ideas-on-how-to-improve-scientific-research-9e2e56474132 https://medium.com/@barmstrong/ideas-on-how-to-improve-scien... [2] https://en.wikipedia.org/wiki/The_Case_Against_Education https://en.wikipedia.org/wiki/The_Case_Against_Education
- cousin_it 6y ago
  > Solution: radically reduce public funding for scientific research and for university education as a whole Wouldn't that make many scientists (both good and bad) move to other countries where funding is easier to get?
  - setgree 6y ago
    Some, perhaps. Others would likely choose different careers, or would adapt. FWIW, and I didn't clarify this enough in my post -- I meant this as an example solution of something that would change the incentives rather than the culture, not necessarily as my own full-throated endorsement; I do personally think that steps in this direction would be for the best, but it's not as though there wouldn't be downsides that would need to be managed/mitigated.
abdullahkhalids 6y ago
I think a lot of scientists would actually do "boring research". (a) they actually know such work is useful (b) some know that they don't produce work as good as the best in the field, but are happy to do more grunt work at lower stress level, rather than endlessly chasing "high impact" publications. Unfortunately, there is no funding for such research. Which is find really sad. Private grants might need to show "impact" but State run grants don't have that many constraints, and they could conceivably offer such grants.
- gvurrdon 6y ago
  I'd happily have done that grunt work, back when I was in a lab. The work may not have been interesting but at least I could feel as if I was being useful whilst also paying the mortgage. Instead, I've ended up writing software which is related to assisting journals in the task of getting researchers to share their data.
  - abdullahkhalids 6y ago
    That sounds fascinating. Can you talk a bit more about your work? What are the principal challenges in getting people to share their data?
    - gvurrdon 6y ago
      No problem. This article might be of some interest: https://www.go-fair.org/fair-principles/ https://www.go-fair.org/fair-principles/ There are, of course, political challenges in persuading researchers to share data, primarily the fear of other researchers using those data for their own papers or to secure funding, depriving the originator of the work of the chance to do the same. I hear this at conferences quite often. The part I'm working on is at attempt to catalogue the places were research data may be stored and the requirements various journals and repositories impose on researchers who want to deposit data there. Gathering this information, curating it and making it easily accessible is not trivial, and relies upon a lot of manual curation by specialist "knowledge engineers".
meow1032 6y ago
> What might work is that we require replication work for a PhD I don't think this will work. All it will do is devalue the value of replication studies because only PHD students do replication studies. It's also not in their best interest especially if they dispute findings of established researchers. Also, we have to get away from the idea that the scientist's job is to think and write, and literally all of the other work can be shuffled off onto low wage (or no wage), low status workers. This is one of the biggest reasons that science is going through such a crisis. If you want enough papers to consistently get grants you probably need at least 4/5 PHD students every few years. This causes a massive glut in the job market. It also dissociates scientists from their work. I've met esteemed computational biologists who could barely work a computer. All of their code was written, run, and analyzed by graduate students or post docs. They were competent enough at statistics, but that level of abstraction from the actual work is troubling.
- jpeloquin 6y ago
  Requiring replication work for a PhD seems like a great idea. PhD programs already use a mandatory exercise—the qualifying exam—to check a student's competence, with ambiguous effectiveness. Turning the qualifying exam into a replication study seems like a win: it tests the student's ability to do their actual job rather than pass an abstract test, and produces output that is useful both to the student and the community. The qualifying exam committee (usually ~ 4 PIs from different labs) can do quality control on the replication. > All it will do is devalue the value of replication studies because only PHD students do replication studies. It's also not in their best interest especially if they dispute findings of established researchers. Most studies are done by students regardless, so it seems unlikely that replication studies would be devalued merely because they're done by students. Although disputing the findings of established researchers can be risky, they would be publishing jointly with their PI (or, with the above implementation, multiple PIs), not alone with no support. Few students want to stay in academia, so it usually doesn't matter to them if a professor at some other institution gets offended. Most importantly, if everyone is doing replication studies, there will be so many disputations flying around that any particular person is less likely to be singled out for retaliation.

searchableguy 6y ago

A more radical solution: Basic income for all researchers and provide efficient digital way to see how funds are being spent like open collective for open source. Most problems arise at an individual level where people don't get paid unless they are putting research papers on the new hype thing and the other is expenditure on middlemen and bureaucracy which isn't easily visible. Eliminate the use of research grants for funding people's living. Bit pedantic but you should use they instead of he/she. People can have other pronouns.

kebman 6y ago
Oh, you mean tenure? UBI is a general thing for the whole population, and not something just for scientists. Tenure is in general a good, as long as scientists keep working, and there are some controls keeping them producing at least "something". And let's face it, most of them do. This is great, because it releaves scientists of pressure, which means more of them can be creative in their research, which actually speeds up discoveries. This is also a valid approach within tech and IT btw. But it should be said that it has a very poor track record outside of these knowledge-heavy professions. On the other hand, if you really think UBI is a good solution, then I'm afraid you don't know how an economy really works.
- searchableguy 6y ago
  Yeah, I meant something like tenure but less restricted for anyone doing research. > On the other hand, if you really think UBI is a good solution, then I'm afraid you don't know how an economy really works. Could you elaborate more?
  - kebman 6y ago
    It has been discussed to death, but for the sake of argument, let's review these two completely opposing factions, who both agree that UBI doesn't work: Zero Hedge on why UBI doesn't work: https://safehaven.com/markets/economy/Why-Universal-Basic-Income-Wont-Work.html https://safehaven.com/markets/economy/Why-Universal-Basic-In... The Guardian on why UBI doesn't work: https://www.theguardian.com/commentisfree/2019/may/06/universal-basic-income-public-realm-poverty-inequality https://www.theguardian.com/commentisfree/2019/may/06/univer...
    - cousin_it 6y ago
      Your second link mentions "universal basic services", I always supported that idea but didn't know it had a name. It's better than UBI because it sidesteps the problem of cost disease (landlords raising rents in response, etc).
      - kebman 6y ago
        Why not go for the original idea, and go straight for Communism instead?
- mattkrause 6y ago
  I think they mean something like UBI: non- or less-competitive grants that would fund modest-sized projects (or pilot studies for larger ones). Tenure doesn't really help with this problem in the lab sciences. While it lets faculty members keep their jobs, it usually doesn't come with enough funding to do experimental work. In fact, while you can "keep" a tenured position without grants, many places find ways to...discourage that (move your office to a shoebox in the sub-basement, crappy teaching and service assignments).
gen155954609803 6y ago
>Bit pedantic but you should use they instead of he/she. People can have other pronouns. One of those other pronouns is literally "they", so you're still risking using the wrong pronoun by using "they" for someone who prefers to be referred to as "he". Was anyone else taught the rule that when writing about a third-party whose gender is unknown, use your own pronouns?
0d9eooo 6y ago
IIRC, Hungary has implemented something like this and it's been floated as an idea, to distribute grant dollars as a fixed amount to everyone who has an H index or citation count above a certain amount. When I've talked about this with colleagues, they've argued this would make problems worse, and it might, but I think the idea was more that once you reach a certain threshold you should be funded some amount. Of course, I'd argue that this is basically the idea behind tenure at an R1 institution, that once you reach a certain level the state or private board of trustees is paying your salary to do research. But nowadays tenure at an R1 institution is proxy for external grant dollars, which defeats the purpose and is redundant. The real problem is indirect funds, which create profit for institutions. The federal government needs to eliminate indirect funds, so that grant dollars aren't perversely incentivised and tenure is based on research quality rather than profit.

asutekku 6y ago

On an unrelated note, just use theirs instead if his/her. It makes the text easier to read and is the grammatically correct way.

Sebb767 6y ago
Thanks for the suggestion! I've edited the question.

setgree 6y ago

Tyler Cowen writes [0] that the most important question in economics, to him, is > how do differences of culture — however defined — interact with traditional economic mechanisms involving prices, incomes, and simple comparative statics? Are those competing explanations, namely cultural vs. economic? Richie's answers are mostly focused on changing the culture of science, and while there are lots of ways we could change the incentives, none of them would be pretty. Example: let's say we want to more closely align research and actionable results, e.g., a product a company can use (Brian Armstrong argues for something like this [1]). Solution: radically reduce public funding for scientific research and for university education as a whole (in line with Bryan Caplan's arguments in "The Case Against Education" [2]). Academics, who would be many fewer in number, would then have to get more of their funding from companies, who (presumably) would: A) guide them towards asking market-relevant questions, and B) have a clear incentive to check the data, re-run the code, etc. -- so that the product they built based on that research didn't flop. I think most people would recoil at this proposal. But that's what comes to mind when I think about fixing the incentives rather than the culture. P.S. Small nitpick: Richie gives an _example_ of a perverse incentive in lieu of a definition. [0] https://marginalrevolution.com/marginalrevolution/2017/01/important-unsolved-problems-field.html https://marginalrevolution.com/marginalrevolution/2017/01/im... [1] https://medium.com/@barmstrong/ideas-on-how-to-improve-scientific-research-9e2e56474132 https://medium.com/@barmstrong/ideas-on-how-to-improve-scien... [2] https://en.wikipedia.org/wiki/The_Case_Against_Education https://en.wikipedia.org/wiki/The_Case_Against_Education

cousin_it 6y ago
> Solution: radically reduce public funding for scientific research and for university education as a whole Wouldn't that make many scientists (both good and bad) move to other countries where funding is easier to get?
- setgree 6y ago
  Some, perhaps. Others would likely choose different careers, or would adapt. FWIW, and I didn't clarify this enough in my post -- I meant this as an example solution of something that would change the incentives rather than the culture, not necessarily as my own full-throated endorsement; I do personally think that steps in this direction would be for the best, but it's not as though there wouldn't be downsides that would need to be managed/mitigated.

abdullahkhalids 6y ago

I think a lot of scientists would actually do "boring research". (a) they actually know such work is useful (b) some know that they don't produce work as good as the best in the field, but are happy to do more grunt work at lower stress level, rather than endlessly chasing "high impact" publications. Unfortunately, there is no funding for such research. Which is find really sad. Private grants might need to show "impact" but State run grants don't have that many constraints, and they could conceivably offer such grants.

gvurrdon 6y ago
I'd happily have done that grunt work, back when I was in a lab. The work may not have been interesting but at least I could feel as if I was being useful whilst also paying the mortgage. Instead, I've ended up writing software which is related to assisting journals in the task of getting researchers to share their data.
- abdullahkhalids 6y ago
  That sounds fascinating. Can you talk a bit more about your work? What are the principal challenges in getting people to share their data?
  - gvurrdon 6y ago
    No problem. This article might be of some interest: https://www.go-fair.org/fair-principles/ https://www.go-fair.org/fair-principles/ There are, of course, political challenges in persuading researchers to share data, primarily the fear of other researchers using those data for their own papers or to secure funding, depriving the originator of the work of the chance to do the same. I hear this at conferences quite often. The part I'm working on is at attempt to catalogue the places were research data may be stored and the requirements various journals and repositories impose on researchers who want to deposit data there. Gathering this information, curating it and making it easily accessible is not trivial, and relies upon a lot of manual curation by specialist "knowledge engineers".

meow1032 6y ago

> What might work is that we require replication work for a PhD I don't think this will work. All it will do is devalue the value of replication studies because only PHD students do replication studies. It's also not in their best interest especially if they dispute findings of established researchers. Also, we have to get away from the idea that the scientist's job is to think and write, and literally all of the other work can be shuffled off onto low wage (or no wage), low status workers. This is one of the biggest reasons that science is going through such a crisis. If you want enough papers to consistently get grants you probably need at least 4/5 PHD students every few years. This causes a massive glut in the job market. It also dissociates scientists from their work. I've met esteemed computational biologists who could barely work a computer. All of their code was written, run, and analyzed by graduate students or post docs. They were competent enough at statistics, but that level of abstraction from the actual work is troubling.

jpeloquin 6y ago
Requiring replication work for a PhD seems like a great idea. PhD programs already use a mandatory exercise—the qualifying exam—to check a student's competence, with ambiguous effectiveness. Turning the qualifying exam into a replication study seems like a win: it tests the student's ability to do their actual job rather than pass an abstract test, and produces output that is useful both to the student and the community. The qualifying exam committee (usually ~ 4 PIs from different labs) can do quality control on the replication. > All it will do is devalue the value of replication studies because only PHD students do replication studies. It's also not in their best interest especially if they dispute findings of established researchers. Most studies are done by students regardless, so it seems unlikely that replication studies would be devalued merely because they're done by students. Although disputing the findings of established researchers can be risky, they would be publishing jointly with their PI (or, with the above implementation, multiple PIs), not alone with no support. Few students want to stay in academia, so it usually doesn't matter to them if a professor at some other institution gets offended. Most importantly, if everyone is doing replication studies, there will be so many disputations flying around that any particular person is less likely to be singled out for retaliation.