2 ms·

> everybody in academia is already aware of those problems (they are personally annoyed by them on a day-to-day basis). But the serious path to fixing them is n

> everybody in academia is already aware of those problems (they are personally annoyed by them on a day-to-day basis). But the serious path to fixing them is not the buzz saw What is the serious path to fixing them? I don't think academics are moving the needle with their current strat of a bit of handwringing followed by conceding to the system anyway.

bee_rider 1y ago
Big picture—I think it is unreasonable to expect us to solve this problem that has plagued a giant community of pretty smart people, in a couple Hackernews comments. But, in the interest of conversation… Replication problems—I guess this is sort of a social problem really, it isn’t seen as prestigious to just try and replicate research. Maybe we should start to hand out more grants for that (we get what we pay for, right?). Perhaps this could also help with “publish or perish,” we could stop using paper published as a figure of merit and start using number of papers reproduced. I would put this in the bucket of insufficient diligence applied by the grant issuers. Overproduction of PhDs, I dunno, if lots of people want to go into research I don’t see that as a huge problem. Rather than overproduction of PhDs, perhaps we have a problem of seeing grad students as cheap labor. I think it is widely understood that the hours that grad students bill are almost entirely bullshit (you spend way more time doing anything than your contract says), so maybe we need to police that better (it seems like the grad students’ and the NSF could easily have aligned interests here, in terms of making sure grad students get paid more). Pay them more, the university will have to employ fewer. However, we’ll probably need to hire more dedicated instructors at that point.
- mike_hearn 1y ago
  It's somewhat irrelevant to this thread because academic CS doesn't have a quality problem on the scale that other fields do, and the NSF cuts aren't directly motivated by paper quality anyway (you could argue the stated motivation traces to the same root causes). But, "replication crisis" is a misnomer. The problem isn't replication, not really. A better name would be the scientific validity crisis. This comes up in every HN discussion of science and academia so I wrote a full explanation of why here: https://blog.plan99.net/replication-studies-cant-fix-science-0e195234a280 https://blog.plan99.net/replication-studies-cant-fix-science... ... but tl;dr many papers that are scientifically invalid will replicate just fine. Even if academics suddenly started directing funding to replication studies on a huge scale, nothing would change and there would be no surge in trust in science.

bee_rider 1y ago

Big picture—I think it is unreasonable to expect us to solve this problem that has plagued a giant community of pretty smart people, in a couple Hackernews comments. But, in the interest of conversation… Replication problems—I guess this is sort of a social problem really, it isn’t seen as prestigious to just try and replicate research. Maybe we should start to hand out more grants for that (we get what we pay for, right?). Perhaps this could also help with “publish or perish,” we could stop using paper published as a figure of merit and start using number of papers reproduced. I would put this in the bucket of insufficient diligence applied by the grant issuers. Overproduction of PhDs, I dunno, if lots of people want to go into research I don’t see that as a huge problem. Rather than overproduction of PhDs, perhaps we have a problem of seeing grad students as cheap labor. I think it is widely understood that the hours that grad students bill are almost entirely bullshit (you spend way more time doing anything than your contract says), so maybe we need to police that better (it seems like the grad students’ and the NSF could easily have aligned interests here, in terms of making sure grad students get paid more). Pay them more, the university will have to employ fewer. However, we’ll probably need to hire more dedicated instructors at that point.

mike_hearn 1y ago
It's somewhat irrelevant to this thread because academic CS doesn't have a quality problem on the scale that other fields do, and the NSF cuts aren't directly motivated by paper quality anyway (you could argue the stated motivation traces to the same root causes). But, "replication crisis" is a misnomer. The problem isn't replication, not really. A better name would be the scientific validity crisis. This comes up in every HN discussion of science and academia so I wrote a full explanation of why here: https://blog.plan99.net/replication-studies-cant-fix-science-0e195234a280 https://blog.plan99.net/replication-studies-cant-fix-science... ... but tl;dr many papers that are scientifically invalid will replicate just fine. Even if academics suddenly started directing funding to replication studies on a huge scale, nothing would change and there would be no surge in trust in science.