3 ms·

While it's noisy and high variance, the approach is similar to start-ups in that we want experiments to be designed in ways that they are likely to confirm unde

by TimPC 11y ago

While it's noisy and high variance, the approach is similar to start-ups in that we want experiments to be designed in ways that they are likely to confirm underlying theories. This makes positive results more valuable than negative results and something the scientist has some influence over, but not complete control of. Successful experiments increase the chance of future experiments being successful in much the same way that having previously had a successful start-up increases the chances of future start-up success. Particularly when committees cover huge areas, it's a lot easier to fund based on track records than to accurately determine experiment quality of an experiment well outside your expertise.

Fede_V 11y ago
I hope I'm not reading what you wrote uncharitably, but, by rewarding people with positive results, we are inevitably encouraging people to either design their experiments in ways that confirm their hypothesis, or at worse, to falsify their data. Thankfully, I've seen very little of the latter, but I've seen quite a bit of the former. Even unconsciously, people are afraid to design experiments that would give a black and white answer and falsify a cherish theory. People with lots of positive results are not necessarily better scientists. They are just as often less scrupulous and better salesmen - Peter Thiel noticed that as well.
learnstats2 11y ago
Yes, and that causes so many problems. As you indicate, all of science is easily biased by what has been done and what has been published in the past. Negative results are essential for a complete understanding of science and when they are repressed, complete access to the globally available data set is not available. If I'm an ethically-dubious pharma or lobby group, I love this situation. I keep funding testing until someone comes up with a positive result by chance or p-value hacking - then fanfare that result all the way to the media and to the bank. (Negative results don't need to be seen) If I'm a cynical scientist, I take the money and p-value hack until I get the results my funding body wants, so that I have access to more money in their next round of funding. Now I'm far more successful than a completely honest scientist. In fact, the honest scientist will either seek to replicate what the cynical scientist did, or leave the profession.

by TimPC 11y ago

Fede_V 11y ago

I hope I'm not reading what you wrote uncharitably, but, by rewarding people with positive results, we are inevitably encouraging people to either design their experiments in ways that confirm their hypothesis, or at worse, to falsify their data. Thankfully, I've seen very little of the latter, but I've seen quite a bit of the former. Even unconsciously, people are afraid to design experiments that would give a black and white answer and falsify a cherish theory. People with lots of positive results are not necessarily better scientists. They are just as often less scrupulous and better salesmen - Peter Thiel noticed that as well.

learnstats2 11y ago

Yes, and that causes so many problems. As you indicate, all of science is easily biased by what has been done and what has been published in the past. Negative results are essential for a complete understanding of science and when they are repressed, complete access to the globally available data set is not available. If I'm an ethically-dubious pharma or lobby group, I love this situation. I keep funding testing until someone comes up with a positive result by chance or p-value hacking - then fanfare that result all the way to the media and to the bank. (Negative results don't need to be seen) If I'm a cynical scientist, I take the money and p-value hack until I get the results my funding body wants, so that I have access to more money in their next round of funding. Now I'm far more successful than a completely honest scientist. In fact, the honest scientist will either seek to replicate what the cynical scientist did, or leave the profession.