4 ms·

It might not be perfect, but it is the best way we have come across.

by cuchoi 9y ago

nonbel 9y ago
What about actual science? Where you figure out how to get reproducible results, come up with theories to explain why those results are the way they are, then make predictions with the theories to test on new data?
- BeetleB 9y ago
  >come up with theories to explain why those results are the way they are, then make predictions with the theories to test on new data? Probably never will happen in social science, and in much of medicine. Too many confounding variables. I'm willing to bet that if we threw away all the "knowledge" we obtained from p-tests, and stuck only to the methodology you espouse, we would know far less true stuff than we do currently (albeit we would not know a lot of false stuff). Simply put: Even though there is a lot of noise, we do know more using these methods. And the nice thing about bad results due to misuse of statistics: Much of it is not reproducible. Ultimately, the problem isn't the use of these statistical methods. The problem is the lack of emphasis on reproducibility.
  - nonbel 9y ago
    >"Probably never will happen in social science, and in much of medicine. Too many confounding variables." This is some kind of talking point. I'm not sure who is behind it, but it is totally false. There are tons of examples of this, especially from the pre-NHST era: https://en.wikipedia.org/wiki/Kleiber%27s_law https://en.wikipedia.org/wiki/Kleiber%27s_law https://en.wikipedia.org/wiki/Law_of_mass_action https://en.wikipedia.org/wiki/Law_of_mass_action http://www.ncbi.nlm.nih.gov/pubmed/17845298 http://www.ncbi.nlm.nih.gov/pubmed/17845298 https://en.wikipedia.org/wiki/Law_of_effect https://en.wikipedia.org/wiki/Law_of_effect http://www.tandfonline.com/doi/pdf/10.1080/00221309.1934.9917847 http://www.tandfonline.com/doi/pdf/10.1080/00221309.1934.991... https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2916857/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2916857/ https://en.wikipedia.org/wiki/Compartmental_models_in_epidemiology https://en.wikipedia.org/wiki/Compartmental_models_in_epidem... http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2007940/pdf/brjcancer00386-0010.pdf http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2007940/pdf/brjc... https://en.wikipedia.org/wiki/Cardiac_output https://en.wikipedia.org/wiki/Cardiac_output
- matt4077 9y ago
  "Actual science" tends to move towards empiricism, not theory-building. See for example the evidence-based medicine movement, or effective altruism. Besides that–how is what you're proposing different from the status quo? Every social program ever was born from a theory, i. e. "education creates agency, and is therefore a better investment than direct money grants". It just happens to be a lot more difficult to run randomised experiments in sociology, because it is (by definition) the study of very large, interconnected systems. Societies just don't lend themselves to the sort of reductionism that gave us all the advances in physics and mathematics.
  - nonbel 9y ago
    "Evidence based medicine" is your best example of science? No, that does not meet the definition I gave. RCT's are not the gold standard of research (how did solar system astronomy get so good with only a couple dozen objects?).

It might not be perfect, but it is the best way we have come across.

by cuchoi 9y ago

It might not be perfect, but it is the best way we have come across.

What about actual science? Where you figure out how to get reproducible results, come up with theories to explain why those results are the way they are, then make predictions with the theories to test on new data?

BeetleB 9y ago
>come up with theories to explain why those results are the way they are, then make predictions with the theories to test on new data? Probably never will happen in social science, and in much of medicine. Too many confounding variables. I'm willing to bet that if we threw away all the "knowledge" we obtained from p-tests, and stuck only to the methodology you espouse, we would know far less true stuff than we do currently (albeit we would not know a lot of false stuff). Simply put: Even though there is a lot of noise, we do know more using these methods. And the nice thing about bad results due to misuse of statistics: Much of it is not reproducible. Ultimately, the problem isn't the use of these statistical methods. The problem is the lack of emphasis on reproducibility.
- nonbel 9y ago
  >"Probably never will happen in social science, and in much of medicine. Too many confounding variables." This is some kind of talking point. I'm not sure who is behind it, but it is totally false. There are tons of examples of this, especially from the pre-NHST era: https://en.wikipedia.org/wiki/Kleiber%27s_law https://en.wikipedia.org/wiki/Kleiber%27s_law https://en.wikipedia.org/wiki/Law_of_mass_action https://en.wikipedia.org/wiki/Law_of_mass_action http://www.ncbi.nlm.nih.gov/pubmed/17845298 http://www.ncbi.nlm.nih.gov/pubmed/17845298 https://en.wikipedia.org/wiki/Law_of_effect https://en.wikipedia.org/wiki/Law_of_effect http://www.tandfonline.com/doi/pdf/10.1080/00221309.1934.9917847 http://www.tandfonline.com/doi/pdf/10.1080/00221309.1934.991... https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2916857/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2916857/ https://en.wikipedia.org/wiki/Compartmental_models_in_epidemiology https://en.wikipedia.org/wiki/Compartmental_models_in_epidem... http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2007940/pdf/brjcancer00386-0010.pdf http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2007940/pdf/brjc... https://en.wikipedia.org/wiki/Cardiac_output https://en.wikipedia.org/wiki/Cardiac_output
matt4077 9y ago
"Actual science" tends to move towards empiricism, not theory-building. See for example the evidence-based medicine movement, or effective altruism. Besides that–how is what you're proposing different from the status quo? Every social program ever was born from a theory, i. e. "education creates agency, and is therefore a better investment than direct money grants". It just happens to be a lot more difficult to run randomised experiments in sociology, because it is (by definition) the study of very large, interconnected systems. Societies just don't lend themselves to the sort of reductionism that gave us all the advances in physics and mathematics.
- nonbel 9y ago
  "Evidence based medicine" is your best example of science? No, that does not meet the definition I gave. RCT's are not the gold standard of research (how did solar system astronomy get so good with only a couple dozen objects?).