7 ms·

It's strange to me that the gate is based on success, and not just a panel of your peers/mentors judging the way you performed the study. After all, negative re

by Sileni 8y ago

It's strange to me that the gate is based on success, and not just a panel of your peers/mentors judging the way you performed the study. After all, negative results are just as valuable as positive results, unless we decide we're going to ignore them and let someone else run the same experiment... which is what we're doing. At least when considering someone for a PhD or Masters, you'd think the point would be to learn good practice, not to actually push the field forward. There's just not as much frontier as when the rules were set.

acchow 8y ago
> After all, negative results are just as valuable as positive results This is not true. It's incredibly easy (and you can be infinitely lazy) to come up with any bad hypothesis that has almost no chance of turning out positive results. Good hypothesis: Existence of Higgs boson at 125 GeV. Bad hypothesis: Eating chocolate in the morning decreases the risk of developing lung cancer. We value positive results more because negative results are incredibly easy to make just by coming up with bad hypotheses.
- dragonwriter 8y ago
  Negative results on “good” hypotheses (by your implicit standard, which seems to be “prior probability of positive results”), though, are still as valuable as positive results (and negative results on hypotheses that seem obvious intuitively are arguably more valuable than positive results.)
  - acchow 8y ago
    agree, but in order to incentivize people to come up with "good hypotheses" we need to devalue negative results. Otherwise everyone can just suck up all the funding for total garbage hypotheses that will almost definitely end up with negative results anyway. "A mutation in the P53AIP1 has a protective effect against alzheimers" Yeah, I totally picked a random gene off wikipedia. This will probably have a negative result. So helpful.
- Sileni 8y ago
  In most cases you have to justify your research already in the form of grant proposals. If we're already vetting the quality of hypotheses, why not rely on that process to support the value of the experiment? Further, I don't have any reason to believe the second is a bad hypothesis because I'm not an oncologist. If an oncologist put forth that hypothesis, I'd expect other oncologists to respond. I wouldn't dismiss it outright. I'd wait to see if it made it through the natural filtering system that is academia. If the results somehow bubbled up through that system, I'd expect them to be somehow meaningful to me. I just feel there's a lack of internal support for academia's own systems, and it's creating a lot of doubled-or-worse work for new scientists.

by Sileni 8y ago

acchow 8y ago

> After all, negative results are just as valuable as positive results This is not true. It's incredibly easy (and you can be infinitely lazy) to come up with any bad hypothesis that has almost no chance of turning out positive results. Good hypothesis: Existence of Higgs boson at 125 GeV. Bad hypothesis: Eating chocolate in the morning decreases the risk of developing lung cancer. We value positive results more because negative results are incredibly easy to make just by coming up with bad hypotheses.

dragonwriter 8y ago
Negative results on “good” hypotheses (by your implicit standard, which seems to be “prior probability of positive results”), though, are still as valuable as positive results (and negative results on hypotheses that seem obvious intuitively are arguably more valuable than positive results.)
- acchow 8y ago
  agree, but in order to incentivize people to come up with "good hypotheses" we need to devalue negative results. Otherwise everyone can just suck up all the funding for total garbage hypotheses that will almost definitely end up with negative results anyway. "A mutation in the P53AIP1 has a protective effect against alzheimers" Yeah, I totally picked a random gene off wikipedia. This will probably have a negative result. So helpful.
Sileni 8y ago
In most cases you have to justify your research already in the form of grant proposals. If we're already vetting the quality of hypotheses, why not rely on that process to support the value of the experiment? Further, I don't have any reason to believe the second is a bad hypothesis because I'm not an oncologist. If an oncologist put forth that hypothesis, I'd expect other oncologists to respond. I wouldn't dismiss it outright. I'd wait to see if it made it through the natural filtering system that is academia. If the results somehow bubbled up through that system, I'd expect them to be somehow meaningful to me. I just feel there's a lack of internal support for academia's own systems, and it's creating a lot of doubled-or-worse work for new scientists.