4 ms·

It would be more than just half productivity. Not only do you have to do the work twice, but you add the delay of someone else replicating before something can

It would be more than just half productivity. Not only do you have to do the work twice, but you add the delay of someone else replicating before something can be published and built upon by others. If you are developer, imagine how much your productivity would drop going from a 3 minute build to a 1 day build.

orangepurple 3y ago
Terrible analogy. It might take months to come up with an idea but another should be able to follow your method and implement it much more quickly than it took you to come up with the concept and implement it.
- magimas 3y ago
  horrible take. Taking the LK99 situation as an example: simply copying and adapting a well described growth recipee to your own setup and lab conditions may take weeks. And how would you address situations where measurement setups only exist once on the earth? How would you do peer replication of LHC measurements? Wait for 50 years till the next super-collider is built and someone else can finally verify the results? On a smaller scale: If you need measurements at a synchrotron radiation source to replicate a measurement, is someone supposed to give up his precious measurement time to replicate a paper he isn't interested in? And is the original author of a paper that's in the queue for peer replication supposed to wait for a year or two till the reviewer gets a beamtime on an appropriate measurement station? Even smaller: I did my PhD in a lab with a specific setup that only a single other group in the world had an equivalent to. You simply would not be able to replicate these results. Peer replication is completely unfeasible in experimental fields of science. The current process of peer review is alright, people just need to learn that single papers standing by themselves don't mean too much. The "peer replication" happens over time anyway when others use the same tools, samples, techniques on related problems and find results in agreement with earlier papers.
- evandrofisico 3y ago
  Usually coming up with a idea is the easy part. For example, in my PhD project, i started with an idea from my advisor that he had in the early 2000. Implementing the code for the simulation and analysis of the data? four months, at most. Running the simulation? almost three years until I had data with good enough resolution for publishing.
  - tnecniv 3y ago
    It’s also very easy to come up with bad ideas — I did plenty of that and I still do, albeit less than I used to. Finding an idea that is novel, interesting, and tractable given your time, skills, resources, and knowledge of the literature is hard, and maybe the most important skill you develop as a researcher. For a reductive example, the idea to solve P vs NP is a great one, but I’m not going to do that any time soon!
    - PaulHoule 3y ago
      I'd say it is a bad idea because you're not going to succeed at it.
      - tnecniv 3y ago
        Sure because that’s an easy example and kind of my point. When you’re starting your PhD, it can be hard to determine what questions you can and cannot answer given your skills and surrounding literature. That comes from experience and trying (and failing) a lot.
- cycomanic 3y ago
  I think you don't understand how much work is involved in just building the techniques and expertise to pull some experiments off (let's not even talk about the equipment). Even if someone meticulously documents their process, it could still take months to replicate the results. I'm familiar with lithography/nanofabrication and I know that it is typically the case that a process developed in one clean-room can not be directly applied to a different clean room and instead one has to develop a new process based on what the other results. Even in the same lab it can often happen that if you come back to a process after a longer time, that things don't work out anymore and quite a bit of troubleshooting ensues (maybe a supplier for some chemical changed and even though it should be the same formula it behaves slightly different).
  - RoyalHenOil 3y ago
    Months. Haha. I previously worked in agricultural research (in the private sector), and we spent YEARS trying to replicate some published research from overseas. And that was research that had previously been successfully replicated, and we even flew in the original scientists and borrowed a number of their PhD students for several months, year after year, to help us try to make it work. We never did get it to fully replicate in our country. We ended up having to make some pretty extreme changes to the research to get similar (albeit less reliable) results here. We never did figure out why it worked in one part of the world but not another, since we controlled for every other factor we could think of (including literally importing the original team's lab supplies at great expense, just in case there was some trace contaminant on locally sourced materials).
    - hnfong 3y ago
      > We never did figure out why it worked in one part of the world but not another Doesn't that indicate further research is needed? It sounds fascinating to me. (I know it isn't interesting for the people who couldn't get it working.) It also might indicate that the original research was incomplete in the sense that it might be a fluke due to specific conditions in the original country which isn't universal.
  - faeriechangling 3y ago
wilde 3y ago
You can easily look up non peer reviewed papers on arxiv. Why would this be different with replication?

orangepurple 3y ago

Terrible analogy. It might take months to come up with an idea but another should be able to follow your method and implement it much more quickly than it took you to come up with the concept and implement it.

magimas 3y ago
horrible take. Taking the LK99 situation as an example: simply copying and adapting a well described growth recipee to your own setup and lab conditions may take weeks. And how would you address situations where measurement setups only exist once on the earth? How would you do peer replication of LHC measurements? Wait for 50 years till the next super-collider is built and someone else can finally verify the results? On a smaller scale: If you need measurements at a synchrotron radiation source to replicate a measurement, is someone supposed to give up his precious measurement time to replicate a paper he isn't interested in? And is the original author of a paper that's in the queue for peer replication supposed to wait for a year or two till the reviewer gets a beamtime on an appropriate measurement station? Even smaller: I did my PhD in a lab with a specific setup that only a single other group in the world had an equivalent to. You simply would not be able to replicate these results. Peer replication is completely unfeasible in experimental fields of science. The current process of peer review is alright, people just need to learn that single papers standing by themselves don't mean too much. The "peer replication" happens over time anyway when others use the same tools, samples, techniques on related problems and find results in agreement with earlier papers.
evandrofisico 3y ago
Usually coming up with a idea is the easy part. For example, in my PhD project, i started with an idea from my advisor that he had in the early 2000. Implementing the code for the simulation and analysis of the data? four months, at most. Running the simulation? almost three years until I had data with good enough resolution for publishing.
- tnecniv 3y ago
  It’s also very easy to come up with bad ideas — I did plenty of that and I still do, albeit less than I used to. Finding an idea that is novel, interesting, and tractable given your time, skills, resources, and knowledge of the literature is hard, and maybe the most important skill you develop as a researcher. For a reductive example, the idea to solve P vs NP is a great one, but I’m not going to do that any time soon!
  - PaulHoule 3y ago
    I'd say it is a bad idea because you're not going to succeed at it.
    - tnecniv 3y ago
      Sure because that’s an easy example and kind of my point. When you’re starting your PhD, it can be hard to determine what questions you can and cannot answer given your skills and surrounding literature. That comes from experience and trying (and failing) a lot.
cycomanic 3y ago
I think you don't understand how much work is involved in just building the techniques and expertise to pull some experiments off (let's not even talk about the equipment). Even if someone meticulously documents their process, it could still take months to replicate the results. I'm familiar with lithography/nanofabrication and I know that it is typically the case that a process developed in one clean-room can not be directly applied to a different clean room and instead one has to develop a new process based on what the other results. Even in the same lab it can often happen that if you come back to a process after a longer time, that things don't work out anymore and quite a bit of troubleshooting ensues (maybe a supplier for some chemical changed and even though it should be the same formula it behaves slightly different).
- RoyalHenOil 3y ago
  Months. Haha. I previously worked in agricultural research (in the private sector), and we spent YEARS trying to replicate some published research from overseas. And that was research that had previously been successfully replicated, and we even flew in the original scientists and borrowed a number of their PhD students for several months, year after year, to help us try to make it work. We never did get it to fully replicate in our country. We ended up having to make some pretty extreme changes to the research to get similar (albeit less reliable) results here. We never did figure out why it worked in one part of the world but not another, since we controlled for every other factor we could think of (including literally importing the original team's lab supplies at great expense, just in case there was some trace contaminant on locally sourced materials).
  - hnfong 3y ago
    > We never did figure out why it worked in one part of the world but not another Doesn't that indicate further research is needed? It sounds fascinating to me. (I know it isn't interesting for the people who couldn't get it working.) It also might indicate that the original research was incomplete in the sense that it might be a fluke due to specific conditions in the original country which isn't universal.
- faeriechangling 3y ago

wilde 3y ago

You can easily look up non peer reviewed papers on arxiv. Why would this be different with replication?