4 ms·

I think there are several issues with this. In these fields, theories (not so fundamental ones) may not be very useful, in contrast with fields like Mathematics

I think there are several issues with this. In these fields, theories (not so fundamental ones) may not be very useful, in contrast with fields like Mathematics or Physics where a hypothesis may lead to a lot of other theoretical results. For example, if material A has property X, there will be a lot of applications for it, but this hypothesis cannot be used as a foundation for other theoretical research. As a result, it is not useful to just state a hypothesis: you have to verify it as well, and negative results are not really valued because it means nothing to others if you rejected something they don't care about.

contravariant 4y ago
Not sure if properties of materials is the best example, those tend to have some theory behind them that, if not explains, at least defines the property and its consequences. If one research team measures the stiffness of a material it's easy enough to make a test that replicates their result. Medical research in particular seems to suffer a lot from papers where something has been measured with no supporting theory whatsoever and consequently no real information other than "If you do what we did you probably get the same result". Rather unsurprisingly this is hard to replicate because there's no theory telling you what is and isn't relevant to replicate. I'm not saying this is easy to fix, but maybe we should focus more effort on papers that expand the theory, rather than papers that at best show that something might be the case but we don't know why.
- watwut 4y ago
  It makes sense to me to first collect small amounts of data, only then formulate theories and only then verify those with larger experiments. Especially when experiments are expensive. It does not make sense to act like small studies are proof of anything the way people like to use those studies on the internet. It is wrong that researchers are effectively punished for reproduction, verification etc. But the existence of small data collection studies on itself is not wrong.

contravariant 4y ago

Not sure if properties of materials is the best example, those tend to have some theory behind them that, if not explains, at least defines the property and its consequences. If one research team measures the stiffness of a material it's easy enough to make a test that replicates their result. Medical research in particular seems to suffer a lot from papers where something has been measured with no supporting theory whatsoever and consequently no real information other than "If you do what we did you probably get the same result". Rather unsurprisingly this is hard to replicate because there's no theory telling you what is and isn't relevant to replicate. I'm not saying this is easy to fix, but maybe we should focus more effort on papers that expand the theory, rather than papers that at best show that something might be the case but we don't know why.

watwut 4y ago
It makes sense to me to first collect small amounts of data, only then formulate theories and only then verify those with larger experiments. Especially when experiments are expensive. It does not make sense to act like small studies are proof of anything the way people like to use those studies on the internet. It is wrong that researchers are effectively punished for reproduction, verification etc. But the existence of small data collection studies on itself is not wrong.