3 ms·
> I haven't, but I'm unclear why it's relevant given that you don't seem to really be disagreeing! It's relevant because you are criticizing the process but yo
by ModernMech 5y ago
> I haven't, but I'm unclear why it's relevant given that you don't seem to really be disagreeing!
It's relevant because you are criticizing the process but you don't seem to understand how it actually works. Your original characterization was that grant money is "this firehose of tax money being sprayed everywhere without even the slightest bit of accountability in how it's used." The reality is, when I get grant money I need to account for how every dollar is spent, and if there are ever any questions about spending, I better have the receipts to back it up. The other reality is, I only get to spend a small fraction of a grant award, as the University takes most of it off the top, and my students take almost all the rest in the form of a tuition remission and a stipend, leaving whatever is left over for equipment and conference costs. Then there are strict conflict of interest regulations which come with their own reporting requirements. I don't even get all of the money at once; I'll get some up front and then I have to show significant midterm progress in order to get more. There's accountability at multiple layers by multiple organizations.
> The NSF denies 80% of grant applications because there is a radical oversupply of people who want to be scientists and the NSF has a finite budget.
It's accountability in the form of: if you didn't do what you promised you'd do, then you don't get any more money and your career is derailed. Isn't that what you want? Anyway, do we have an oversupply of scientists relative to the amount of science that needs to be done? I don't think so. The NSF budget is finite, but it's also embarrassingly miniscule given the upside of research that has come out of NSF funded projects.
> If it existed it would look like this ... We will therefore prosecute you for research fraud and failure to meet the terms of your contract.
Fraud is one thing. I'm not going to say we shouldn't prosecute fraud. But treating a grant proposal like a contract with positive deliverables (no negative results allowed) is the exact problem. Research is not development. Research implies failure. You can't do one without the other. Failure to meet stated objectives shouldn't be met with prosecution. That just further incentivizes fraud.
If there's a replicability crises it just means we need to spend money on replication research. Researchers know no one is going to bother replicating their study because there is no grant money available for redoing someone else's research. Grant agencies don't pay for that kind of thing, and you can't make a career doing that kind of research. No one gets tenure this way. If we want studies to be replicated, we need to allocate money to replicate them, and we need to incentivize people to do so by making it a viable career for a Ph.D.
> Lots of people, non scientists included, have high expectations for results placed upon them. The right mechanisms and incentives to stop fraud are not simply having low expectations of scientists, that's absurd and wouldn't be seriously proposed as a solution in any other area of society.
I didn't say we should have low expectations, I said we should have realistic expectations, and yes, that does involve lowering expectations from where they are right now. Because the current expectation is this: you have to write a proposal that has a <20% chance of getting funding. If you can't get that funding your career is basically over, so you better promise the world, because everyone else is. In this proposal you need to lay out a research plan for the next 3-5 years and you have to convince the funding agency that your research is going to change the world as we know it. If within that time you fail to meet your stated objectives, you will find funding hard to come across, and your tenure will be threatened, meaning you will probably lose your job and have to move your family. On top of that you want to add potential federal prosecution to stakes, thinking that will make things better.
> Expectations on scientists are already rock bottom .. The solution to fraud is tighter contracts to ensure the rules are clear, and systematic prosecutions of people who break them.
Okay, run with this idea: exactly what rules need to be clearer and exactly how do the contracts need to be tightened? Because there are already clear rules and tight contracts, yet the problem persists. Will clearer rules and tighter contracts fix it? How?
I'll tell you what I think will happen with this system: you'll chase out all of the public scientists because the stakes are too high. Already the pay is too good on the corporate side, and now you add potential federal prosecution to the list if I don't meet positive deliverables? No thanks. I'll go work for Microsoft where my research will be privatized. You might be okay with this as you pointed out you believe a profit motive is good for research, but you know who wouldn't be good with this? Microsoft. And Google. And all the other tech companies who were (or will be) built on top of technologies that started as government funded research. All this does is make Microsoft stronger. Is that what we want? What about the next Microsoft or Google? Where will they come from?
I'll give you a concrete example of where your idea fails: the 2004 DARPA Grand Challenge. Millions were spent trying to bootstrap autonomous cars, and what was the result? They all crashed, no one completed the race. What should the response have been, to prosecute everyone involved? No, they tried again and gave everyone more money. Next time around in 2005 more succeeded (mostly because they relaxed the expectations).
Then in 2007 we saw the first real demonstration of autonomous cars in the DARPA Urban Challenge. Today, everything Tesla, Google, GM, Ford, et al. are doing with driverless cars is based on the research that happened in 2004-2007. Without government funded autonomous car research, there would be no Tesla or Waymo today. That's how research works, you try, you fail, you try again, and you have no idea how far your impact will be, and really no one does. If we try to control this process toward producing only successes with contracts and positive deliverables, like it's an engineering project (with prosecution of failure and all), it just means we're going to lose dynamics like the Grand Challenges, and the broader economy will suffer for it.
Take all that money you want to invest in prosecutors, courts, lawyers, and prisons, and invest that in a system where replication studies are well funded and a viable career path for scientists. Increase funding into the NSF and other grant funding agencies to hire more people to consider grants, and increase grant throughput. I guarantee you you'll fix a lot of the problems you're identifying.
- native_samples 5y agoI think we are 80% in agreement but still using words differently. > when I get grant money I need to account for how every dollar is spent Yes I know, but that's not what I mean by accountability. Again: nobody is upset with academics because of expenses scandals or taking too many expensive flights. Well, except maybe for climatologists who supposedly take more flights than the average academic, but that's due to the perception of hypocrisy rather than concern over cost. People are getting upset because when they download and read papers, the papers turn out to be bad and there are no visible consequences for that. Even just getting a clearly fraudulent paper retracted is reported to be a nightmare, according to people who search for scientific fraud as a hobby like Elizabeth Bik. And I've read endless reams of terrible papers that were useless or outright deceptive, I tried reporting a few and nobody ever cared. Now, you're arguing that there is accountability of the following form: > It's accountability in the form of: if you didn't do what you promised you'd do, then you don't get any more money This is true given that scientists are promising the NSF to publish papers, not strictly speaking to do research, and therefore by implication promising to come up with interesting claims, not necessarily true claims. But that's not what we want. This is an inevitable problem with government funding of research. The buyer, the government, cannot really check if the claims they're buying from scientists are true, so they need proxies like did it get published, did it get cited, etc. But those aren't the same things. Corporate research doesn't have this problem because the corporate will try to apply the research at some point and if it was fraudulent they will discover it at that point, and of course they're strongly incentivized to ensure it never gets to that point in the first place. In theory the government could write grants in such a way that money is awarded independent of what claims end up being made, instead awarding money for the quality of work done. That's what you're arguing for here. And indeed corporate labs write contracts in this exact way. Scientists get a salary in a corporate lab, they don't have to write grants. They do have to convince their management chain that the research is worth funding, but there are many different ways to do that which don't involve continually publishing astonishing claims in scientific journals. You're asking me to propose how science should work instead but, indeed, you already know my answer: eliminate the NSF completely, and stop subsidizing student loans. All science should be funded by companies. They have already solved the problems you're treating as novel / intractable above. Scientists are awarded salaries and promotions by firms on a more flexible basis than the NSF. Importantly, they are rewarded for doing research not producing claims. Companies can do this because they have management structures sufficiently well staffed to closely monitor what scientists are doing. That means if a firm is truly committed to research then the scientists will get paid even if their programme has some dry years. Plus there's a huge body of law handling fraud and corruption in the workplace. At the same time, firms are incentivized to eliminate the research that is probably always going to be nearly useless. Outside of firms selling books or self help courses I doubt many would subsidize sociology or gender studies for example, and it's also unclear that would be a loss. Your argument about who it would or wouldn't be good for seems a bit contradictory and I struggled to follow it. You're arguing it would be both bad for Google and Microsoft yet also make them stronger. I disagree with both possibilities: I think they would hardly notice the difference and it wouldn't affect how powerful they are. Having worked for one of those companies and also worked at a startup where we often read research papers in a certain subfield of CS with views to maybe applying them, my view is that even in the relatively good field of computer science, most academic output is useless and has no impact. These firms do not rely heavily on government funded research: - The web was very briefly funded for a couple of years as a side project of CERN, but then R&D was taken over by the private sector where it remained ever since. Page & Brin never even finished their PhD before moving their research into the private sector! It's hardly a mystery where the next Google will come from - probably the same place the previous one did, a garage in Silicon Valley. - What government funded tech was Microsoft built on? The internet? Microsoft is still with us in spite of the internet, not because of it! Or are you going back to military computers in World War 2? Military R&D is different, governments can fund that semi-effectively because they actually use the outputs. - Neural networks were a backwater until Jeff Dean resurrected the field using the resources of the private sector, academia has been chasing to catch up ever since. There are a lot of other examples. The DARPA Grand Challenge is not an example of what I'm talking about because: 1. DARPA is military research and therefore structured differently to how the NSF does things. The very structure of it as a Grand Challenge is a clue here: the output of the programme was cars (not) going round a track, not papers and citations. 2. I'm not arguing for prosecution of researchers who end up with null results!