4 ms·

I'm also in physics and I agree with this. There's no chance that 80% is wrong. One factor contributing to this. In natural sciences, you take other people's p

I'm also in physics and I agree with this. There's no chance that 80% is wrong. One factor contributing to this. In natural sciences, you take other people's papers as truth and build on that. In theoretical physics on the other hand the _first step_ is you reproduce their results.

stevenjgarner 4y ago
Why do you think that is? (Serious question). My understanding in life science research is that as a rule, papers do not get reproduced, as there is limited benefit to the researcher. Why is the same not true in physics? What is so seriously flawed about life science research that is not flawed about physics?
- martincmartin 4y ago
  My guess: In theoretical physics, you reproduce by spending an afternoon manipulating equations. Maybe a few days at most. But you reproduce in life science you need to get test subjects and treatments, etc. For example, how would you reproduce that a diet high in olive oil reduces the chance of heart attack?
  - t6jvcereio 4y ago
    It's usually a lot more than "a few days at most". In my field it was often between a week to a couple of months. But you got the right idea.
- jltsiren 4y ago
  It depends on the type of research. If your research is mathematical or involves a lot of engineering, you are building on earlier results. In many cases, your own results will depend on the correctness of earlier results in a measurable way. You end up replicating others' research without even trying. Empirical research is harder. You cite earlier results, but there is no clear connection between their correctness and your results. Especially if effect size is small. Earlier research has more effect on the framework you use to interpret your results than on the results. Academic researchers rarely replicate others' results, because it's expensive and not particularly interesting. People typically come to the academia because they want to work on something they personally find interesting. If you want to have experienced scientists working on something administrators tell them to do, you better pay industry-level salaries.
- t6jvcereio 4y ago
  Isn't it obvious? In theoretical physics, one of a researcher's main concern is to not to make wrong claims. Granted this risk aversion has its own downsides, but the upside is that he is extremely careful on what he takes to be true. So it's not true that not reproducing the research you rely on has limited benefit. It has the huge benefit that the researcher convinces himself he's not introducing wrong premises. Research is already difficult enough without making mistakes in this silly way.
patrick451 4y ago
There's limited value in reproducing a theoretical result. That doesn't tell us if it's actually a valid model of the world. You can "reproduce" the results of string theory all day long, but nobody know if that's how things actually work. It's entirely possible that 100% of the papers in that sub-field are wrong even if the math checks out.

stevenjgarner 4y ago

Why do you think that is? (Serious question). My understanding in life science research is that as a rule, papers do not get reproduced, as there is limited benefit to the researcher. Why is the same not true in physics? What is so seriously flawed about life science research that is not flawed about physics?

martincmartin 4y ago
My guess: In theoretical physics, you reproduce by spending an afternoon manipulating equations. Maybe a few days at most. But you reproduce in life science you need to get test subjects and treatments, etc. For example, how would you reproduce that a diet high in olive oil reduces the chance of heart attack?
- t6jvcereio 4y ago
  It's usually a lot more than "a few days at most". In my field it was often between a week to a couple of months. But you got the right idea.
jltsiren 4y ago
It depends on the type of research. If your research is mathematical or involves a lot of engineering, you are building on earlier results. In many cases, your own results will depend on the correctness of earlier results in a measurable way. You end up replicating others' research without even trying. Empirical research is harder. You cite earlier results, but there is no clear connection between their correctness and your results. Especially if effect size is small. Earlier research has more effect on the framework you use to interpret your results than on the results. Academic researchers rarely replicate others' results, because it's expensive and not particularly interesting. People typically come to the academia because they want to work on something they personally find interesting. If you want to have experienced scientists working on something administrators tell them to do, you better pay industry-level salaries.
t6jvcereio 4y ago
Isn't it obvious? In theoretical physics, one of a researcher's main concern is to not to make wrong claims. Granted this risk aversion has its own downsides, but the upside is that he is extremely careful on what he takes to be true. So it's not true that not reproducing the research you rely on has limited benefit. It has the huge benefit that the researcher convinces himself he's not introducing wrong premises. Research is already difficult enough without making mistakes in this silly way.

patrick451 4y ago

There's limited value in reproducing a theoretical result. That doesn't tell us if it's actually a valid model of the world. You can "reproduce" the results of string theory all day long, but nobody know if that's how things actually work. It's entirely possible that 100% of the papers in that sub-field are wrong even if the math checks out.