3 ms·

> This looks an awful lot like survivor bias. I'm not sure it's survivorship bias. H. pylori, for example, actually is a major cause of gastric ulcers. If we

> This looks an awful lot like survivor bias. I'm not sure it's survivorship bias. H. pylori, for example, actually is a major cause of gastric ulcers. If we were actually good at identifying good ideas, Marshall and Warren would have had a pretty steady rise to fame over the course of their careers since they had, in fact, figured it out. For what I've heard, they did not--they were treated pretty shabbily for a while, then the field did an about-face and they won a Nobel prize. It's not the case that anyone who (somehow) survived that long gets a Nobel! > I doubt you want to do away with peer review. I don't think anyone wants to completely do away with the idea of having others sanity-check their research. However, there is a huge gap between peer review as an abstract concept and peer review as currently practiced in NIH study sections. Personally, I think that there's a lot of room for improvement in the latter. For example, you could imagine giving program managers more room to actually manage a portfolio of research and (e.g.) hedge across multiple hypotheses.

tertius 9y ago
> I'm not sure it's survivorship bias. I'm referring to good ideas being underfunded for a time. Only good ideas survive. Bad ideas obviously don't get to a point where they turn into something accepted and widely applicable (I'm sure there's a countercase or two...) So only the good survive. Does that mean we need to fund across the board? That's just not feasible, so we have some good ideas that go unfunded, but a whole lot of bad ideas that do not. > manage a portfolio of research and (e.g.) hedge across multiple hypotheses. I see this happening in well funded labs. Taking a bit of money from here and some from there in things that are novel, but ethically related enough to be able to be justified. But the ethics involved makes this dangerous. You bring up good points.

tertius 9y ago

> I'm not sure it's survivorship bias. I'm referring to good ideas being underfunded for a time. Only good ideas survive. Bad ideas obviously don't get to a point where they turn into something accepted and widely applicable (I'm sure there's a countercase or two...) So only the good survive. Does that mean we need to fund across the board? That's just not feasible, so we have some good ideas that go unfunded, but a whole lot of bad ideas that do not. > manage a portfolio of research and (e.g.) hedge across multiple hypotheses. I see this happening in well funded labs. Taking a bit of money from here and some from there in things that are novel, but ethically related enough to be able to be justified. But the ethics involved makes this dangerous. You bring up good points.