3 ms·

I think you don't understand how much work is involved in just building the techniques and expertise to pull some experiments off (let's not even talk about the

I think you don't understand how much work is involved in just building the techniques and expertise to pull some experiments off (let's not even talk about the equipment). Even if someone meticulously documents their process, it could still take months to replicate the results. I'm familiar with lithography/nanofabrication and I know that it is typically the case that a process developed in one clean-room can not be directly applied to a different clean room and instead one has to develop a new process based on what the other results. Even in the same lab it can often happen that if you come back to a process after a longer time, that things don't work out anymore and quite a bit of troubleshooting ensues (maybe a supplier for some chemical changed and even though it should be the same formula it behaves slightly different).

RoyalHenOil 3y ago
Months. Haha. I previously worked in agricultural research (in the private sector), and we spent YEARS trying to replicate some published research from overseas. And that was research that had previously been successfully replicated, and we even flew in the original scientists and borrowed a number of their PhD students for several months, year after year, to help us try to make it work. We never did get it to fully replicate in our country. We ended up having to make some pretty extreme changes to the research to get similar (albeit less reliable) results here. We never did figure out why it worked in one part of the world but not another, since we controlled for every other factor we could think of (including literally importing the original team's lab supplies at great expense, just in case there was some trace contaminant on locally sourced materials).
- hnfong 3y ago
  > We never did figure out why it worked in one part of the world but not another Doesn't that indicate further research is needed? It sounds fascinating to me. (I know it isn't interesting for the people who couldn't get it working.) It also might indicate that the original research was incomplete in the sense that it might be a fluke due to specific conditions in the original country which isn't universal.
faeriechangling 3y ago
You are making an argument for replication not against it by stressing how meticulously documenting your process is an “if”, strewing how things like a supplier for a chemical can change and render reproduction impossible, and even if an observation can only be replicated in one clean room that means you effectively only have as long as that clean room remains opens to replicate it. You are almost stressing all the ways we are producing garbage rendered non-reproducible with deficient documentation of processes, changes in supply, and changes in the environment. All three can be minimized through peer replication.
- wholinator2 3y ago
  Well there isn't a way to fix this replicative difficulty for many fields. So you're suggesting that the theoretical understanding gained from having produced a specific material just never be added to scientific understanding at all, ever. In an ideal world with ideal budget allotment there would be time and money to build entire replicating labs that can be moved and modularly reconfigured with absolute control over air particulates, pressure, flow, sufficient to replicate any room possible anywhere on earth. But if a lab in the himalayas was able to make a magnet that fundamentally reconfigures our understanfing of physical phenomena, we should simply abandon that potential deeper truth into were able to build a second lab in the Himalayas, or in the middle of the Atlantic. Reproduction is hard, really really fucking hard. Just saying, that means we should replicate before trying to understand, means essentially cutting off the understanding. And like others have said, if someone wants to build from it they'll depend on that information being correct, if no one can manage to ever build from it then the idea dies. Also there's a huge difference between, replicate this study on infant response to stimulus, or spider colony behavior, and, replicate this incredibly intricate semiconductor that took years of configuration to correctly produce.

RoyalHenOil 3y ago

Months. Haha. I previously worked in agricultural research (in the private sector), and we spent YEARS trying to replicate some published research from overseas. And that was research that had previously been successfully replicated, and we even flew in the original scientists and borrowed a number of their PhD students for several months, year after year, to help us try to make it work. We never did get it to fully replicate in our country. We ended up having to make some pretty extreme changes to the research to get similar (albeit less reliable) results here. We never did figure out why it worked in one part of the world but not another, since we controlled for every other factor we could think of (including literally importing the original team's lab supplies at great expense, just in case there was some trace contaminant on locally sourced materials).

hnfong 3y ago
> We never did figure out why it worked in one part of the world but not another Doesn't that indicate further research is needed? It sounds fascinating to me. (I know it isn't interesting for the people who couldn't get it working.) It also might indicate that the original research was incomplete in the sense that it might be a fluke due to specific conditions in the original country which isn't universal.

faeriechangling 3y ago

You are making an argument for replication not against it by stressing how meticulously documenting your process is an “if”, strewing how things like a supplier for a chemical can change and render reproduction impossible, and even if an observation can only be replicated in one clean room that means you effectively only have as long as that clean room remains opens to replicate it. You are almost stressing all the ways we are producing garbage rendered non-reproducible with deficient documentation of processes, changes in supply, and changes in the environment. All three can be minimized through peer replication.

wholinator2 3y ago
Well there isn't a way to fix this replicative difficulty for many fields. So you're suggesting that the theoretical understanding gained from having produced a specific material just never be added to scientific understanding at all, ever. In an ideal world with ideal budget allotment there would be time and money to build entire replicating labs that can be moved and modularly reconfigured with absolute control over air particulates, pressure, flow, sufficient to replicate any room possible anywhere on earth. But if a lab in the himalayas was able to make a magnet that fundamentally reconfigures our understanfing of physical phenomena, we should simply abandon that potential deeper truth into were able to build a second lab in the Himalayas, or in the middle of the Atlantic. Reproduction is hard, really really fucking hard. Just saying, that means we should replicate before trying to understand, means essentially cutting off the understanding. And like others have said, if someone wants to build from it they'll depend on that information being correct, if no one can manage to ever build from it then the idea dies. Also there's a huge difference between, replicate this study on infant response to stimulus, or spider colony behavior, and, replicate this incredibly intricate semiconductor that took years of configuration to correctly produce.