3 ms·
In principle this sort of technique seems like something that could benefit any sort of science, including climate science, immensely. The number of studies wit
by gpm 9y ago
In principle this sort of technique seems like something that could benefit any sort of science, including climate science, immensely. The number of studies with unreproducible, and likely flat out wrong results, is scarily high. (And yes, there is certainly real critique that can happen in climate science, not on whether or not climate change exists at this point, but on details of certain papers, and even topics like "how realistic is the clathrate gun hypothesis".)
Unfortunately in practice this likely ends up being the government sponsoring the same sort of bullshit that the oil companies have been putting out for ages, and not real critique. Red team/blue team is also a unfortunate name given the political connotations...
- KirinDave 9y agoI'd be more optimistic if Pruitt hadn't flat out said his plan was to discredit everything...
- unabst 9y agoThis. We need to change the world so such statements immediately discredit the people who made them in a consequential manner. They already do for some people, but not where it matters. We're already in such a world when it comes to racism and sexual harassment (Clipper's owner, Fox news anchors). We're just not there yet for political integrity, an oxymoron of sorts.
- vannevar 9y ago"And yes, there is certainly real critique that can happen in climate science..." The kind of critique that routine happens within the scientific community, over time. Scientists are highly motivated to prove the majority wrong, because that's how you make your career, by finding something new that everyone else missed. By contrast, the EPA 'critique' will be a political charade, a circus of crackpots and shills assembled by the administration to rationalize its predetermined policy decisions.
- gpm 9y agoIf that kind of critique was happening routinely we wouldn't have 50% of scientists not being able to reproduce their own paper [0]. The sad truth is that incentives have aligned so that sufficient critique doesn't happen, and that showing some paper doesn't reproduce isn't going to make your career. Government funding to encourage such critique would be great. I don't disagree on this likely be(com)ing a political charade. [0] http://www.nature.com/news/1-500-scientists-lift-the-lid-on-reproducibility-1.19970 http://www.nature.com/news/1-500-scientists-lift-the-lid-on-...
- trendia 9y ago> 50% of scientists not being able to reproduce their own paper In the case of the psychology replication crisis (note that I'm NOT talking about climate science here), the problem was the result of a few factors: 1. the lack of incentives to replicate others' work 2. the use of p-values to determine what gets published 3. the low probability of a true positive These factors are not part of the critique process of peer review. That is, reviewers are looking at whether the methodology is valid. "Did the scientists follow proper procedures? Do the results match up with what they claim?" While these questions can identify methodological errors, they cannot identify when an experiment is a statistical fluke. Suppose the true positive rate is 5% and the false positive rate is 5%. That means that only 10% of experiments will give results that suggest publishing. But if you look at published papers only, the probability that the paper is false is not 5%, it's 50%! The solution to this problem is not to throw out peer review, it's to fix the three issues above. We must create a system that incentivizes replication. Perhaps 1st-year grad students will be required to attempt to replicate existing work. (This would be more interesting for the grad student than taking an additional class!) Perhaps we should move away from p-values (but what do we replace that with?) However, the replication crisis is not really the problem of bad science or corruption -- it's mostly the result of bad incentives and false positives.
- pas 9y agoP-values have the feature as being "simple", but here's the/a better/Bayesian approach to hypothesis testing: https://stats.stackexchange.com/a/20393 https://stats.stackexchange.com/a/20393