3 ms·

If research moves more to arxiv + discussion forums rather than paid journals, sharing research results earlier seems like a good thing, not bad. If they didn't

by vasco 3y ago

If research moves more to arxiv + discussion forums rather than paid journals, sharing research results earlier seems like a good thing, not bad. If they didn't publish early, when they published maybe things would be clearer for other researchers. But by publishing earlier other people have the choice to engage with it or wait for later. Seems like many people decided to not wait, so it seems like it was a good choice?

was_a_dev 3y ago
I think this would vary greatly on the field. This example of results is one of the better examples imo
brutusborn 3y ago
I think this is the best direction for science generally: publish everything open and early. It might be naive, but I would love an environment where everyone self published or arxiv papers/reviews/datasets/models. Then journals could publish the best papers/review preprints. I imagine a world where journals beg researchers for the rights to publish their paper after seeing an excellent preprint. Or a journal finds an interesting comment on PubPeer and pays for the author to expand it into a more detailed review.
- WanderPanda 3y ago
  Nice counterfactual! Do you think it is just a path dependency that we didn‘t end up in the (local) minimum that you mention?
  - brutusborn 3y ago
    Yes I think we will end there someday but are still overcoming a publishing model historically based on physical journals. Open science becoming more common shows that progress is occurring. It will take a while before academics and systems upgrade skills and infrastructure. When they do, I am excited for enhanced capabilities like continuous science. Imagine a website with a study result with P values which continuously update as data is published and integrated into the existing analysis! https://towardsdatascience.com/continuous-science-6b5ed833ab23 https://towardsdatascience.com/continuous-science-6b5ed833ab...
    - bckr 3y ago
      You’ve heard of CI/CD, now we bring you Continuous Experimentation and Publishing
ryukoposting 3y ago
This is a great philosophy in software, where meaningful contributions can be made by any given person with a laptop. It's not as compelling when the only people who can meaningfully advance the research need to spend millions in grant money and/or allocate time to use expensive equipment that's shared across other research groups.
- ryukoposting 3y ago
  Just want to add to this thought: this isn't to say I believe the status quo in academia isn't antiquated. It is. My point is that we can't just "take open source and move it over there," so to speak.
- brutusborn 3y ago
  What is the difference? I would argue it is more important to publish early when using shared expensive equipment, since there is a huge opportunity cost to wasting resources when better information exists.
- bckr 3y ago
  But in what case is this the reality? Science is generally dealing with more data than any one person can look at. Anyone with a laptop can provide unique analyses of the data which can fuel new hypotheses, which can be explored with simulations and further analysis. the big machines represent a bottleneck only in the physical experiments that they enable, but a lot of science can be done around that. Tighten the feedback loops by all means I say Especially for public science, but that’s another story
matthewdgreen 3y ago
This is an incredibly high-profile type of result, so in this case people are willing to spend a lot of resources replicating this result, even if the paper appears half-baked and unfinished. Most scientific results aren’t like that: people will be enthusiastic about polished work but won’t waste any time on something half-baked. Think about how many “Show HN” posts don’t get voted up to the front page. Now imagine there is no HN and no voting system to rank posts, and also you didn’t have anyone test your code so your cool project immediately crashes on 50% of browsers.

by vasco 3y ago

was_a_dev 3y ago

I think this would vary greatly on the field. This example of results is one of the better examples imo

brutusborn 3y ago

I think this is the best direction for science generally: publish everything open and early. It might be naive, but I would love an environment where everyone self published or arxiv papers/reviews/datasets/models. Then journals could publish the best papers/review preprints. I imagine a world where journals beg researchers for the rights to publish their paper after seeing an excellent preprint. Or a journal finds an interesting comment on PubPeer and pays for the author to expand it into a more detailed review.

WanderPanda 3y ago
Nice counterfactual! Do you think it is just a path dependency that we didn‘t end up in the (local) minimum that you mention?
- brutusborn 3y ago
  Yes I think we will end there someday but are still overcoming a publishing model historically based on physical journals. Open science becoming more common shows that progress is occurring. It will take a while before academics and systems upgrade skills and infrastructure. When they do, I am excited for enhanced capabilities like continuous science. Imagine a website with a study result with P values which continuously update as data is published and integrated into the existing analysis! https://towardsdatascience.com/continuous-science-6b5ed833ab23 https://towardsdatascience.com/continuous-science-6b5ed833ab...
  - bckr 3y ago
    You’ve heard of CI/CD, now we bring you Continuous Experimentation and Publishing

ryukoposting 3y ago

This is a great philosophy in software, where meaningful contributions can be made by any given person with a laptop. It's not as compelling when the only people who can meaningfully advance the research need to spend millions in grant money and/or allocate time to use expensive equipment that's shared across other research groups.

ryukoposting 3y ago
Just want to add to this thought: this isn't to say I believe the status quo in academia isn't antiquated. It is. My point is that we can't just "take open source and move it over there," so to speak.
brutusborn 3y ago
What is the difference? I would argue it is more important to publish early when using shared expensive equipment, since there is a huge opportunity cost to wasting resources when better information exists.
bckr 3y ago
But in what case is this the reality? Science is generally dealing with more data than any one person can look at. Anyone with a laptop can provide unique analyses of the data which can fuel new hypotheses, which can be explored with simulations and further analysis. the big machines represent a bottleneck only in the physical experiments that they enable, but a lot of science can be done around that. Tighten the feedback loops by all means I say Especially for public science, but that’s another story

matthewdgreen 3y ago

This is an incredibly high-profile type of result, so in this case people are willing to spend a lot of resources replicating this result, even if the paper appears half-baked and unfinished. Most scientific results aren’t like that: people will be enthusiastic about polished work but won’t waste any time on something half-baked. Think about how many “Show HN” posts don’t get voted up to the front page. Now imagine there is no HN and no voting system to rank posts, and also you didn’t have anyone test your code so your cool project immediately crashes on 50% of browsers.