5 ms·

How often is a study even repeated in the course of normal scientific research? I think most studies are focused on expanding the research and therefore knowle

by rplst8 8y ago

How often is a study even repeated in the course of normal scientific research? I think most studies are focused on expanding the research and therefore knowledge about the science being studied. Accepting previous studies as fact is dangerous and could easily lead to a house of cards scenario. I think this is especially true in the very specific, niche studies that are common in today's highly competitive graduate student and publish or die research landscape.

lotsofpulp 8y ago
That is why, as a (global) society, we should spend our resources in making sure the data we have is accurate by funding research and repeated studies, and that data should be open to all to verify and build upon.
- jbob2000 8y ago
  I think we're passed the point where studies can be reproduced effectively. I actually think the scientific model is falling apart. We've discovered the easy things, and those studies were easy to reproduce. We built a model of science around the low hanging fruit. How are you going to reproduce a study about a cancer drug? Synthesizing the drug itself is a huge endeavor, never mind finding a group of participants with the exact cancer you need. How would you reproduce the Large Hadron Collider experiments? You'd need to rebuild one of the most technically challenging, most expensive scientific instruments ever made.
  - SubiculumCode 8y ago
    Cost aside you make a good point. In today's science we are trying to uncover subtle effects with much more noise.
  - lotsofpulp 8y ago
    There's enough resources to conduct multiple wars and bailout the financial industry every 10 years, but not enough resources to educate our fellow citizens and fund research? (with all results open and accessible to all through this fancy thing called the internet) Our problem isn't lack of resources. There's 7.5 billion people in the world, I can't believe we've ran out of control groups and experiments to do. Maybe it's a little more difficult with medical edge cases, but there's plenty of other things to research. It's just hard to do it politically, especially with people that are "anti-science", meaning against funding research that might show outcomes that aren't good for them economically, or against their religion.
  - haZard_OS 8y ago
    > I actually think the scientific model is falling apart. You must have a totally different definition of "the scientific model" than I do. I suspect (although I could be wrong) is that you have an oversimplified view of just what the scientific process is. The scientific model/process is not a linear list of discrete steps moving from "Ask a question" to "Communicate results"; it is not a discrete and static set of criteria we apply to individual experiments or instrument output. Berkeley University does a good job of providing visualizations on this topic: http://undsci.berkeley.edu/article/0_0_0/howscienceworks_02 http://undsci.berkeley.edu/article/0_0_0/howscienceworks_02
    - jbob2000 8y ago
      Ok, I read your link. I still think my comments stands. My argument would be that the "Community Analysis and Feedback" section is practically impossible to maintain. Back in early science days, this used to be a valuable section because the pool of "scientists" was huge and they often crossed disciplines. Now you have all sorts of specialties such that the pool of colleagues who are able to understand and assess your work might be, like, 8 people. There was a Japanese mathematician who recently put out a huge proof that claimed to solve a major math problem (I am forgetting all of the important details here, so forgive me). The news wasn't so much that he created the proof, but that they couldn't find anyone to verify it!
SubiculumCode 8y ago
Most scientists in my profession look for converging evidence from multiple lines. While the same study is rarely repeated, the same concept does get tested multiple times with different approaches.
gus_massa 8y ago
There are a lot of "new" articles that are just a small tweak of a previous article with a lot of hype to make it appear as a groundbreaking result. So there is a lot of implicit informal reproduction. You can't trust blindly the previous studies because they may be wrong, or have hidden assumptions. A well known case is the use of a control group. If you want to study a new drug, you must compare the results of the patients with a similar group that is taking the previous drug and/or a group that is taking no drug. You can't just trust that the cure rate of the previous study is 23.573% and compare it with your results. (And there are some trick to get more reliable results, like using a placebo instead of no drug, and doing a double blind study.)
UncleMeat 8y ago
Explicitly replicated, rarely. But other similar studies will either reinforce or call into question the original result. This has a similar effect as replication over time.

by rplst8 8y ago

lotsofpulp 8y ago

That is why, as a (global) society, we should spend our resources in making sure the data we have is accurate by funding research and repeated studies, and that data should be open to all to verify and build upon.

jbob2000 8y ago
I think we're passed the point where studies can be reproduced effectively. I actually think the scientific model is falling apart. We've discovered the easy things, and those studies were easy to reproduce. We built a model of science around the low hanging fruit. How are you going to reproduce a study about a cancer drug? Synthesizing the drug itself is a huge endeavor, never mind finding a group of participants with the exact cancer you need. How would you reproduce the Large Hadron Collider experiments? You'd need to rebuild one of the most technically challenging, most expensive scientific instruments ever made.
- SubiculumCode 8y ago
  Cost aside you make a good point. In today's science we are trying to uncover subtle effects with much more noise.
- lotsofpulp 8y ago
  There's enough resources to conduct multiple wars and bailout the financial industry every 10 years, but not enough resources to educate our fellow citizens and fund research? (with all results open and accessible to all through this fancy thing called the internet) Our problem isn't lack of resources. There's 7.5 billion people in the world, I can't believe we've ran out of control groups and experiments to do. Maybe it's a little more difficult with medical edge cases, but there's plenty of other things to research. It's just hard to do it politically, especially with people that are "anti-science", meaning against funding research that might show outcomes that aren't good for them economically, or against their religion.
- haZard_OS 8y ago
  > I actually think the scientific model is falling apart. You must have a totally different definition of "the scientific model" than I do. I suspect (although I could be wrong) is that you have an oversimplified view of just what the scientific process is. The scientific model/process is not a linear list of discrete steps moving from "Ask a question" to "Communicate results"; it is not a discrete and static set of criteria we apply to individual experiments or instrument output. Berkeley University does a good job of providing visualizations on this topic: http://undsci.berkeley.edu/article/0_0_0/howscienceworks_02 http://undsci.berkeley.edu/article/0_0_0/howscienceworks_02
  - jbob2000 8y ago
    Ok, I read your link. I still think my comments stands. My argument would be that the "Community Analysis and Feedback" section is practically impossible to maintain. Back in early science days, this used to be a valuable section because the pool of "scientists" was huge and they often crossed disciplines. Now you have all sorts of specialties such that the pool of colleagues who are able to understand and assess your work might be, like, 8 people. There was a Japanese mathematician who recently put out a huge proof that claimed to solve a major math problem (I am forgetting all of the important details here, so forgive me). The news wasn't so much that he created the proof, but that they couldn't find anyone to verify it!

SubiculumCode 8y ago

Most scientists in my profession look for converging evidence from multiple lines. While the same study is rarely repeated, the same concept does get tested multiple times with different approaches.

gus_massa 8y ago

There are a lot of "new" articles that are just a small tweak of a previous article with a lot of hype to make it appear as a groundbreaking result. So there is a lot of implicit informal reproduction. You can't trust blindly the previous studies because they may be wrong, or have hidden assumptions. A well known case is the use of a control group. If you want to study a new drug, you must compare the results of the patients with a similar group that is taking the previous drug and/or a group that is taking no drug. You can't just trust that the cure rate of the previous study is 23.573% and compare it with your results. (And there are some trick to get more reliable results, like using a placebo instead of no drug, and doing a double blind study.)

UncleMeat 8y ago

Explicitly replicated, rarely. But other similar studies will either reinforce or call into question the original result. This has a similar effect as replication over time.