3 ms·

This is basically "Do preregistration and put it in a blockchain-like structure". It's not a bad idea. It adds cryptographic integrity to good scientific pract

by hannob 8y ago

This is basically "Do preregistration and put it in a blockchain-like structure". It's not a bad idea. It adds cryptographic integrity to good scientific practice. But hardly anyone outside medicine does hypothesis preregistration yet. In computer science it's basically unknown to do it. All the A/B testing you heard about in user studies is almost certainly p-hacked. So the first step would be to convince people that they actually have to do something about p-hacking.

ianbooker 8y ago
Most fields are not accustomed to confirmatory statistical analysis as medicine, psychology or finance are. The poor understanding of p-hacking is a mere consequence of this for most fields. A good custom and first step for any field would be to elaborate more on the validity of an analysis per se, or to give studies a "reality check" at all.
RosanaAnaDana 8y ago
I mean, dealing with the multitude of irreducible results in the natural sciences is a major source of what I would consider to be very bad science, owing to institutionalized biases formed from 'conclusions' made from previously unwarranted results. While it as well is a major issue in medicine, no one dies when some ones claims of a soil carbon characteristic or a stomatal conductance trait end up being irreducible; in-fact it can cost you a career to even open up the can of worms which is validating the previous results on which your work claims to be based. There are two issues at odds here as I see it. One, science is a career, and if you don't show results, well, it will be a short career indeed. Two, science is a philosophy which governs the determination of the best `truth` possible given the evidence. The two are highly at odds, and perhaps this may give a way to at least resolve some issues related to irreconcilability in all science.
Derek_MK 8y ago
>But hardly anyone outside medicine does hypothesis preregistration yet. And that's where this idea fails from a practical standpoint. This system gives scientists the means to do something that hurts their chances at publishing/funding. Now, if it were somehow enforced, then you'd have a different situation, and it could give some serious benefits.

by hannob 8y ago

ianbooker 8y ago

Most fields are not accustomed to confirmatory statistical analysis as medicine, psychology or finance are. The poor understanding of p-hacking is a mere consequence of this for most fields. A good custom and first step for any field would be to elaborate more on the validity of an analysis per se, or to give studies a "reality check" at all.

RosanaAnaDana 8y ago

I mean, dealing with the multitude of irreducible results in the natural sciences is a major source of what I would consider to be very bad science, owing to institutionalized biases formed from 'conclusions' made from previously unwarranted results. While it as well is a major issue in medicine, no one dies when some ones claims of a soil carbon characteristic or a stomatal conductance trait end up being irreducible; in-fact it can cost you a career to even open up the can of worms which is validating the previous results on which your work claims to be based. There are two issues at odds here as I see it. One, science is a career, and if you don't show results, well, it will be a short career indeed. Two, science is a philosophy which governs the determination of the best `truth` possible given the evidence. The two are highly at odds, and perhaps this may give a way to at least resolve some issues related to irreconcilability in all science.

Derek_MK 8y ago

>But hardly anyone outside medicine does hypothesis preregistration yet. And that's where this idea fails from a practical standpoint. This system gives scientists the means to do something that hurts their chances at publishing/funding. Now, if it were somehow enforced, then you'd have a different situation, and it could give some serious benefits.