3 ms·
Well, the smaller your p, the higher your chance of false negatives, too; maybe there was an actual effect there that was worth more study, and you dismissed it
by geofft 9y ago
Well, the smaller your p, the higher your chance of false negatives, too; maybe there was an actual effect there that was worth more study, and you dismissed it.
The problem is that a study that's significant at p = 0.05 but not p = 0.01 is worth something to science. It indicates something worth doing several more studies on, preferably by different teams from different institutes with non-overlapping confounding factors. But the practice of science gives you only two options once you find such a study, as a young scientist: promote it immediately and become visible for having discovered something, or aggressively ignore it and go find something else that you think is likely to be immediately promising. "Doing science," as you put it, is the worst option for you because it ensures you spend a lot more time on something for no reward -- and the rational self-interest here is perfectly rational, because if you don't get funding or tenure, you're not going to do any science any more at all. (As seems to have happened with Dr. Cuddy.)
To be clear, I have no well-formed opinion on whether Dr. Cuddy is a good scientist, having only read this article. I assume she is, or at least that she's honest in her desires. But I think talented, honest people are nonetheless pressured by incentives, and fixing incentives is better for society (assuming we as a larger society wish our scientists to continue to do science) than assuming that people will be self-sacrificial and altruistic in pursuit of a tiny chance at accomplishing the greater good.
- killjoywashere 9y agoHere's the structural problem I see: there are acres, literally, acres of computer science types in SF and certain other enclaves, slaving away on ads when they would rather be curing cancer. But they don't know the first thing about cancer. Meanwhile, there are enclaves where biologists are slaving away capturing data and not even realizing it's data. "Bookkeeping data" they might call it. There should be mixers between the biologists and the computer scientists. Especially non-academic physicians and computer scientists. Look at Kaiser-Permanente, Sutter Health, or Defense Health Agency. Huge orgs, very little research. Data falling off the shelves and they don't know what to do with it.
- nonbel 9y agoPlease no mixers. The AI/ML/CS people should start their own different, better approach.
- slavik81 9y agoMozilla Science runs the global science sprint each year to bring scientists of all kinds together with developers. I bet there's lots of projects that could still use help. https://github.com/mozilla/global-sprint/issues https://github.com/mozilla/global-sprint/issues
- amazingman 9y agoYou’re not wrong. Especially not about the ads. What we need, unfortunately, are (at least) 2 very difficult things to achieve legislatively: 1. A legal framework for health-related products & services that meaningfully protects individual privacy, and yet also preserves the conditions for the kind of velocity and rate of change that brings technological progress to the masses reliably and quickly. 2. Massive academic funding for more “pure” research in hardware & software design, engineering, the philosophy of what’s possible, the phenomenon and ethics of what we choose to build, and so on. TLDR Pay more people more money to do important research in the public domain, which I can’t help but think would attract many of the CS types you mentioned.
- killjoywashere 9y ago> a legal framework that protects privacy yet brings technological progress to the masses quickly That exists, but there's upfront educational costs that need to be handled first. > Massive academic funding Look at Europe: their acedemics work much more closely with industry. Get a someone who can code in a room with a physician, and I garuntee they can find a problem to work on. Show a successful prototype, and we're way below "minimum viable product" here. Show a graph with real data and money will start looking for you. Computer scientists, mathematicians, physicists, and engineers need to go to biology conferences. And biologists should be going to math, ML, and software conferences.
- nonbel 9y ago>"The problem is that a study that's significant at p = 0.05 but not p = 0.01 is worth something to science. It indicates something worth doing several more studies on" I still don't understand this one. The null model is (almost) always false. In this situation a small p-value is only a matter of sample size and measurement precision. How can it possibly indicate you should work more on something? One thing I think may make sense is to get a bunch of p-values and then choose the lowest 10 or whatever to focus on for further work. There is no reason to include an arbitrary cut-off here though.
- nonbel 9y ago>"Well, the smaller your p, the higher your chance of false negatives, too; maybe there was an actual effect there that was worth more study, and you dismissed it." Also noticing this now... No, this is completely untrue. The p-value is not an error rate.