3 ms·

So many problems in science could be alleviated if we changed the culture and processes to multi replication. Your paper can only be accepted into a paper if an

So many problems in science could be alleviated if we changed the culture and processes to multi replication. Your paper can only be accepted into a paper if an unrelated group can replicate the results with only the paper and associated submitted info.

ska 3y ago
You don't really need that (and it would be too expensive), you just need to fix the incentives so that people do more replication.
ffgjgf1 3y ago
Someone will need to pay that other group though. Sometimes quite a lot.
- Voultapher 3y ago
  I'd argue it's worth it. Some practical ideas, when allocating the money for research 30% or so has to be kept back for replication. Public science is mostly government funded. And the government could allocate X% percent of the budget for replication work. It sounds like a solvable problem to me.
  - ffgjgf1 3y ago
    I agree in principle but forcing institutions or governments to allocate money for this and enforce is pretty much an unsolvable problem at this point IMO.
    - Voultapher 3y ago
      Yeah, as I said it's also a question of culture and that's very difficult to change.
doctorpangloss 3y ago
It is a misconception that replication doesn’t occur. In vanguard basic science, every journey crosses replicating the most recent results.
- Voultapher 3y ago
  I'd like to learn more, could you please provide some sources. Based on the recent Ranga Dias fraud, I'd think replication culture still has a long way to go.
- coolhand2120 3y ago
  Is that true? Do you have any citation? It sounds incredible. Every study?
  - pcrh 3y ago
    It's broadly true that most research is eventually subject to attempts at replication. However the replication isn't explicitly attempting to reproduce the exact prior research, but to build on it. If the foundation established by the prior research is flawed, attempts to build on it will usually fail.
tnecniv 3y ago
This doesn’t scale. Sometimes only a small number of labs have the equipment needed to run an experiment, let alone the training on the equipment to do what needs to be done. This equipment can cost millions. Even if the results are correct, experimental science is very finicky by its very nature. Getting an experiment to work can take months.
- Voultapher 3y ago
  I'd say if no one can afford to, or knows how to replicate the thing you did. What point is there in it? And only because such an approach has downsides it's not clear it means the overall benefit of lessening the replication crisis and forcing scientists to provide enough information their work can be replicated at all would be a huge win. Based on people like https://www.youtube.com/@AppliedScience https://www.youtube.com/@AppliedScience experience, 9 out of 10 times there is some crucial information missing in the papers they are trying to replicate.
  - anonymouskimmer 3y ago
    > I'd say if no one can afford to Because sometimes they can afford to build on your prior results, and advance the state of the art. Factoring in all of the malfeasance, what's the trade off between not publishing due to inability/unwillingness to replicate, and publishing bad results? We should know this tradeoff before significantly changing the status quo. > or knows how to replicate the thing you did. Agree completely.
- somenameforme 3y ago
  Like what? I can think of extremely few domains that any decent research university isn't well equipped to competently replicate. One of the very few benefits of the tuition explosion - even undergrads get relatively casual access to equipment worth millions of dollars. Of course they can't replicate things like ultra high energy particle research, but these sort of obscure things make up a very negligible chunk of all science produced, even if it's quite an important little chunk.
  - 3y ago
hcks 3y ago
If this had been enacted, there would never have been any scientific progress. Should tell you something
- Voultapher 3y ago
  You base this claim on what exactly?

ska 3y ago

You don't really need that (and it would be too expensive), you just need to fix the incentives so that people do more replication.

ffgjgf1 3y ago

Someone will need to pay that other group though. Sometimes quite a lot.

Voultapher 3y ago
I'd argue it's worth it. Some practical ideas, when allocating the money for research 30% or so has to be kept back for replication. Public science is mostly government funded. And the government could allocate X% percent of the budget for replication work. It sounds like a solvable problem to me.
- ffgjgf1 3y ago
  I agree in principle but forcing institutions or governments to allocate money for this and enforce is pretty much an unsolvable problem at this point IMO.
  - Voultapher 3y ago
    Yeah, as I said it's also a question of culture and that's very difficult to change.

doctorpangloss 3y ago

It is a misconception that replication doesn’t occur. In vanguard basic science, every journey crosses replicating the most recent results.

Voultapher 3y ago
I'd like to learn more, could you please provide some sources. Based on the recent Ranga Dias fraud, I'd think replication culture still has a long way to go.
coolhand2120 3y ago
Is that true? Do you have any citation? It sounds incredible. Every study?
- pcrh 3y ago
  It's broadly true that most research is eventually subject to attempts at replication. However the replication isn't explicitly attempting to reproduce the exact prior research, but to build on it. If the foundation established by the prior research is flawed, attempts to build on it will usually fail.

tnecniv 3y ago

This doesn’t scale. Sometimes only a small number of labs have the equipment needed to run an experiment, let alone the training on the equipment to do what needs to be done. This equipment can cost millions. Even if the results are correct, experimental science is very finicky by its very nature. Getting an experiment to work can take months.

Voultapher 3y ago
I'd say if no one can afford to, or knows how to replicate the thing you did. What point is there in it? And only because such an approach has downsides it's not clear it means the overall benefit of lessening the replication crisis and forcing scientists to provide enough information their work can be replicated at all would be a huge win. Based on people like https://www.youtube.com/@AppliedScience https://www.youtube.com/@AppliedScience experience, 9 out of 10 times there is some crucial information missing in the papers they are trying to replicate.
- anonymouskimmer 3y ago
  > I'd say if no one can afford to Because sometimes they can afford to build on your prior results, and advance the state of the art. Factoring in all of the malfeasance, what's the trade off between not publishing due to inability/unwillingness to replicate, and publishing bad results? We should know this tradeoff before significantly changing the status quo. > or knows how to replicate the thing you did. Agree completely.
somenameforme 3y ago
Like what? I can think of extremely few domains that any decent research university isn't well equipped to competently replicate. One of the very few benefits of the tuition explosion - even undergrads get relatively casual access to equipment worth millions of dollars. Of course they can't replicate things like ultra high energy particle research, but these sort of obscure things make up a very negligible chunk of all science produced, even if it's quite an important little chunk.
- 3y ago

hcks 3y ago

If this had been enacted, there would never have been any scientific progress. Should tell you something

Voultapher 3y ago
You base this claim on what exactly?