3 ms·

My dad was a geneticist at Harvard and we lived across the street from Stephen J. Gould. When people on Facebook are having a conversation about how Benghazi is

My dad was a geneticist at Harvard and we lived across the street from Stephen J. Gould. When people on Facebook are having a conversation about how Benghazi is a sign of the second coming of Christ, I sometimes take a moment and question my own beliefs. The foundation of my beliefs isn't because someone said punctuated equilibrium is a thing but that tests can replicated and peer review scientific research can stand the test of time.

m_mueller 10y ago
I don't think peer review is one of the foundational ingredients. Scientific work should be repeatable and falsifiable, IMO that is all that really matters. These two properties are already hard to muster though.
- derefr 10y ago
  In centuries past, with small experiments, "peer review" actually was replication—your peers would go and do the experiment themselves and confirm your finding. Insofar as experiments are now too large and costly to allow for "I'll just go check that in my lab tonight"-type replication, peer review is increasingly just a status game.
  - saalweachter 10y ago
    The most valuable thing about peer-reviewed journals isn't that a handful of chumps proof-read the articles before publication but that the journals are widely circulated and read by the appropriate set of scientists and technicians. One of two things will happen after publication. The paper will have no impact, no one will base any further work on it, no one will cite it, and it will vanish like a fart in the wind. In this case, the author gets minimal prestige, and can claim a publication in a maybe-prestigious journal with an embarrassingly small number of follow-up citations. No fame, money, or power for the author, and no impact (negative or positive, if the paper was right or wrong) on the world. Or the paper can spark a lot of ideas in the people who read it. They'll try to build on it, or they'll try to use a technique from it in another application. When this happens, if the paper was solid, then the original author gets citations, speaking engagements, money and power. If the paper was less than solid, and the people trying to build on it keep having trouble, that is when the original author's house of cards comes crashing down. You don't need to directly replicate an experiment when you can build on it. If your experiments building on the original experiment make sense and give consistent results, you've just validated the original research in another way. If your experiments don't, and after weeks and months of wasting time checking your own work and making sure you're not doing wrong, you slowly reach the conclusion that the original research was bogus, then you begin to get angry. Then you begin to tear into the original author and his research, publicly denounce him for wasting your and everyone else's time. Then science self-corrects. So science is generally OK -- either bogus research that gets published has no impact (no different that a lot of good research :-P ), or it proves false when other people try to build on it. Who gets into trouble, is science journalists and laypeople who read newly published articles and assume they are gospel.
    - derefr 10y ago
      You're talking about the kind of sciences with applied-science subfields, where experiments form a natural hierarchy with new knowledge necessarily building upon old. Fields like psychology, on the other hand, need replications—"assuming another result as axiomatic in your own experiment" just doesn't come up much.
      - saalweachter 10y ago
        Why do you think researchers in the various subfields of psychology do not build on each others' work? What do you think they are doing when they read each others' papers and cite them?
- jnicholasp 10y ago
  Peer review as currently done certainly isn't foundational to the scientific method, but the general principle of having ideas checked and examined by as many other minds as are capable of understanding them, does lead to stronger and more reliable ideas over time. Not everything that science deals with is quantifiable or directly testable; results must be interpreted, and theories have to be made about the wider reasons for those results in order to come up with the next set of experiments; this is the realm of discussion and argument, rather than measurement or direct tests, and science is best advanced when there is strong, productive discussion among competent arguers.

m_mueller 10y ago

I don't think peer review is one of the foundational ingredients. Scientific work should be repeatable and falsifiable, IMO that is all that really matters. These two properties are already hard to muster though.

derefr 10y ago
In centuries past, with small experiments, "peer review" actually was replication—your peers would go and do the experiment themselves and confirm your finding. Insofar as experiments are now too large and costly to allow for "I'll just go check that in my lab tonight"-type replication, peer review is increasingly just a status game.
- saalweachter 10y ago
  The most valuable thing about peer-reviewed journals isn't that a handful of chumps proof-read the articles before publication but that the journals are widely circulated and read by the appropriate set of scientists and technicians. One of two things will happen after publication. The paper will have no impact, no one will base any further work on it, no one will cite it, and it will vanish like a fart in the wind. In this case, the author gets minimal prestige, and can claim a publication in a maybe-prestigious journal with an embarrassingly small number of follow-up citations. No fame, money, or power for the author, and no impact (negative or positive, if the paper was right or wrong) on the world. Or the paper can spark a lot of ideas in the people who read it. They'll try to build on it, or they'll try to use a technique from it in another application. When this happens, if the paper was solid, then the original author gets citations, speaking engagements, money and power. If the paper was less than solid, and the people trying to build on it keep having trouble, that is when the original author's house of cards comes crashing down. You don't need to directly replicate an experiment when you can build on it. If your experiments building on the original experiment make sense and give consistent results, you've just validated the original research in another way. If your experiments don't, and after weeks and months of wasting time checking your own work and making sure you're not doing wrong, you slowly reach the conclusion that the original research was bogus, then you begin to get angry. Then you begin to tear into the original author and his research, publicly denounce him for wasting your and everyone else's time. Then science self-corrects. So science is generally OK -- either bogus research that gets published has no impact (no different that a lot of good research :-P ), or it proves false when other people try to build on it. Who gets into trouble, is science journalists and laypeople who read newly published articles and assume they are gospel.
  - derefr 10y ago
    You're talking about the kind of sciences with applied-science subfields, where experiments form a natural hierarchy with new knowledge necessarily building upon old. Fields like psychology, on the other hand, need replications—"assuming another result as axiomatic in your own experiment" just doesn't come up much.
    - saalweachter 10y ago
      Why do you think researchers in the various subfields of psychology do not build on each others' work? What do you think they are doing when they read each others' papers and cite them?
jnicholasp 10y ago
Peer review as currently done certainly isn't foundational to the scientific method, but the general principle of having ideas checked and examined by as many other minds as are capable of understanding them, does lead to stronger and more reliable ideas over time. Not everything that science deals with is quantifiable or directly testable; results must be interpreted, and theories have to be made about the wider reasons for those results in order to come up with the next set of experiments; this is the realm of discussion and argument, rather than measurement or direct tests, and science is best advanced when there is strong, productive discussion among competent arguers.