4 ms·

It's worth noting that this is talking about research that involves sampling groups of people, or other similar sampling approaches. It is not talking about mat

by tensor 8y ago

It's worth noting that this is talking about research that involves sampling groups of people, or other similar sampling approaches. It is not talking about mathematical results which you can often find in computer science, physics, and elsewhere. Not is it necessarily true all of the physical sciences. It also shouldn't be read as "science is broken and wrong so therefore my opinion should be considered equally." There is definitely a problem with accuracy in many scientific fields that needs to be addressed, but the baby doesn't need to be thrown out with the bath water.

bad_user 8y ago
While I agree with the general sentiment that not all science is equal, it’s important to remember that math isn’t science. Science is empirical and involves usage of the _scientific method_ for acquisition of knowledge and math is disqualified. Of course math sits at the foundation of the sciences we are talking about. And truly _proving_ something involves math. That’s not what we are talking about though. And computer science is just like math. It’s not empirical, it’s not characterized by the scientific method. It’s not science either.
- yesenadam 8y ago
  Beware of bugs in the above code; I have only proved it correct, not tried it. – Donald Knuth How did people learn about the nature of, say, 1D cellular automata, or the Mandelbrot set? By experimenting, classifying, observing, testing hypotheses etc.
epistasis 8y ago
It's not even talking about biology research, or even most medically related research. While it was a great point to make at the time, far too much has been made of it. Yes, don't p-hack, but it's also better to publish data than to withold it just because there wasn't a positive result. We need better publication mechanisms for data that doesn't have any significant findings.
- astazangasta 8y ago
  Err, this 100% applies to biology research, where there is absolutely a problem of small sample sizes, small effects, etc. Our studies (my field is cancer) are routinely vastly underpowered because of the high variability of the datasets we are studying, the low availability of samples, and the large number of variables we collect on these sample. Also, to a good approximation, everybody p-hacks. Furthermore, the habit of publishing "noteworthy" results (true in every journal, especially true for large impact factor journals) is essentially p-hacking across the entire field. This is a huge problem.
  - epistasis 8y ago
    How many of your papers consist of a single statistical test, without validation experiments? Because that's the setup that's described in order for "most published research findings are false." In reality, I've never seen a biology paper without several lab techniques and orthogonal verifications, with p-values on some but not all of those experiments.
- serghiou 8y ago
  Good point - the problem is that the grand majority of researchers publish only a very small part of what they actually did - this creates a missing data problem and greatly skews perceptions of those reading their work. If everyone published everything they had done, the problem with reproducibility would have been greatly reduced.

by tensor 8y ago

bad_user 8y ago

While I agree with the general sentiment that not all science is equal, it’s important to remember that math isn’t science. Science is empirical and involves usage of the _scientific method_ for acquisition of knowledge and math is disqualified. Of course math sits at the foundation of the sciences we are talking about. And truly _proving_ something involves math. That’s not what we are talking about though. And computer science is just like math. It’s not empirical, it’s not characterized by the scientific method. It’s not science either.

yesenadam 8y ago
Beware of bugs in the above code; I have only proved it correct, not tried it. – Donald Knuth How did people learn about the nature of, say, 1D cellular automata, or the Mandelbrot set? By experimenting, classifying, observing, testing hypotheses etc.

epistasis 8y ago

It's not even talking about biology research, or even most medically related research. While it was a great point to make at the time, far too much has been made of it. Yes, don't p-hack, but it's also better to publish data than to withold it just because there wasn't a positive result. We need better publication mechanisms for data that doesn't have any significant findings.

astazangasta 8y ago
Err, this 100% applies to biology research, where there is absolutely a problem of small sample sizes, small effects, etc. Our studies (my field is cancer) are routinely vastly underpowered because of the high variability of the datasets we are studying, the low availability of samples, and the large number of variables we collect on these sample. Also, to a good approximation, everybody p-hacks. Furthermore, the habit of publishing "noteworthy" results (true in every journal, especially true for large impact factor journals) is essentially p-hacking across the entire field. This is a huge problem.
- epistasis 8y ago
  How many of your papers consist of a single statistical test, without validation experiments? Because that's the setup that's described in order for "most published research findings are false." In reality, I've never seen a biology paper without several lab techniques and orthogonal verifications, with p-values on some but not all of those experiments.
serghiou 8y ago
Good point - the problem is that the grand majority of researchers publish only a very small part of what they actually did - this creates a missing data problem and greatly skews perceptions of those reading their work. If everyone published everything they had done, the problem with reproducibility would have been greatly reduced.