4 ms·

As a former (PhD) researcher, this is spot on. Here are the factors I've observed: - fear of getting scooped: there's a general fear that sharing data makes it

by axg11 4y ago

As a former (PhD) researcher, this is spot on. Here are the factors I've observed: - fear of getting scooped: there's a general fear that sharing data makes it easier for competing labs to publish on the same topics, leading to the publishing lab "missing out" on publications - fear of being wrong: sharing the data leads to more scrutiny, leading to a higher chance of a fatal flaw being found - extra work: publishing data often involves jumping through extra hoops with little to no personal gain

Hermitian909 4y ago
Not a researcher, but have worked closely with them on data sharing. I think there's a variant of getting scooped worth pointing out, which is that in some fields people are scared of being scooped by a low quality lab that fakes the results. I had a very senior materials science researcher describe what he thought was a promising line of study get shut down because a low quality lab in china scooped him with results good enough for publication but not good enough that people would want to replicate. He was 100% confident their data was faked but getting grant money once that experiment was done was like pulling teeth and so he abandoned the route for other ideas.
- axg11 4y ago
  This is interesting. Not something that I ever came across but can very much believe this happens. Most problems in academia come down to bad incentives. Researchers are incentivised to publish for prestige and citations, which are poor proxies for improving our understanding of the world around us.
  - alexb_ 4y ago
    All problems, and really all actions, can be explained by incentives. Once you understand that people will never under any circumstances do something that both takes effort and is opposite to their incentives, you understand why nearly every problem occurs and why every action is taken.
    - MereInterest 4y ago
      This statement is either so broad that it is obviously false, or has so many implied additional conditions that its application is incredibly narrow. * The time horizon used to judge incentives vary person to person. * Willing martyrs exist for various causes. Therefore, it is not just the incentives that matter, but the belief about those incentives. * Self-sacrifice for others exists (e.g. a mother rushing into a burning building for the chance of saving her children). Therefore, it is not just the incentives that matter, but also the subjective weighting of conflicting incentives. * A person may spend a great deal of time researching betting patterns for playing craps, none of which can overcome the house's advantage. Therefore, it is not just the incentives that matter, but the knowledge of the outcomes. * Somebody may work in an oppressive environment for their entire life, rather than changing careers. Therefore, it is not just the effort required that matters, but also the relative amount of effort on different time horizons. While I agree that incentives are the best way to move population-level behavior, and is usually a good starting point for understanding individual behavior, your categorical statement doesn't have those caveats.
  - andi999 4y ago
    Billion dollar question is what a better proxy would be.
    - mike_hearn 4y ago
      Possibly a reformed patent system, e.g. one in which time limits are variable or in which you don't get a full monopoly (forced licensing terms). At one point I was researching an alternative patent system in which you don't get a monopoly on the idea, but rather for every next licensee they have to pay in a share of the total value of the patent such that e.g. the first licensee pays half which goes direct to the original filer, the next licensee pays a third and that money is split between the other two licensees, the next pays a quarter etc. So the patent gets cheaper over time to license, but you lose the first mover advantages. The problem with that system is that the "value" would have to be determined in some independent manner, e.g. an audited cost+ basis, which wouldn't be reflective of the true value of the idea. Granting time-limited monopoly rights allows the idea to be priced via normal market mechanisms, at the cost of high transaction costs. At any rate, you get the idea. The justification for academic research is that the private sector, supposedly, won't do long term basic science and that grant recipients will. That's a very dubious set of assumptions. We see basic science being done by private labs all over the place, most obviously in AI but also in many other areas of computer science, we see it in the biotech sector, we see it in semiconductor physics etc. The areas where there's no private investment tend also to be the most controversial areas of academia where many people have concluded that the results are systematically useless e.g. social sciences. And meanwhile what we see amongst grant recipients is systematically unscientific behaviour like knowingly publishing claims that aren't actually true, refusing to share data even after promising to do so, accepting unvalidated modelling as 'science' and a million other things.
- jlduan 4y ago
  This doesn't make sense. Being scooped means other people published the same idea before you. If he truly believes the released data were fake, he can for sure publish his results. It’s not unusual for scientists to have different views on certain subjects. Some journals even have specific discussion sections for contradictory results. Finding the truth and correcting scientific records is also an achievement. And at least, in the filed of biology/medicine, there are journals accepting “second first” papers, for example https://elifesciences.org/articles/30076 https://elifesciences.org/articles/30076 . And quit a lot of publications are not grand breaking ideas. Incremental improvements can also be published. I don’t understand your friend’s rationale to abandon the whole project he already started and didn’t want to find the truth and prove his theory (maybe it’s a different field thing).
  - IanCal 4y ago
    > If he truly believes the released data were fake, he can for sure publish his results. You need the funding to get to that point though, and since it wasn't "new" they weren't able to get funded to research it. That's the meaning I got from the comment.
  - Hermitian909 4y ago
    > I don’t understand your friend’s rationale to abandon the whole project he already started and didn’t want to find the truth and prove his theory (maybe it’s a different field thing). He has lots of ideas he wants to try, and he ranks them. The incentives for following this particular idea were reduced by the actions of the other lab, so it dropped in the rankings and he pursued one of the more highly ranked ones.

by axg11 4y ago

Hermitian909 4y ago

Not a researcher, but have worked closely with them on data sharing. I think there's a variant of getting scooped worth pointing out, which is that in some fields people are scared of being scooped by a low quality lab that fakes the results. I had a very senior materials science researcher describe what he thought was a promising line of study get shut down because a low quality lab in china scooped him with results good enough for publication but not good enough that people would want to replicate. He was 100% confident their data was faked but getting grant money once that experiment was done was like pulling teeth and so he abandoned the route for other ideas.

axg11 4y ago
This is interesting. Not something that I ever came across but can very much believe this happens. Most problems in academia come down to bad incentives. Researchers are incentivised to publish for prestige and citations, which are poor proxies for improving our understanding of the world around us.
- alexb_ 4y ago
  All problems, and really all actions, can be explained by incentives. Once you understand that people will never under any circumstances do something that both takes effort and is opposite to their incentives, you understand why nearly every problem occurs and why every action is taken.
  - MereInterest 4y ago
    This statement is either so broad that it is obviously false, or has so many implied additional conditions that its application is incredibly narrow. * The time horizon used to judge incentives vary person to person. * Willing martyrs exist for various causes. Therefore, it is not just the incentives that matter, but the belief about those incentives. * Self-sacrifice for others exists (e.g. a mother rushing into a burning building for the chance of saving her children). Therefore, it is not just the incentives that matter, but also the subjective weighting of conflicting incentives. * A person may spend a great deal of time researching betting patterns for playing craps, none of which can overcome the house's advantage. Therefore, it is not just the incentives that matter, but the knowledge of the outcomes. * Somebody may work in an oppressive environment for their entire life, rather than changing careers. Therefore, it is not just the effort required that matters, but also the relative amount of effort on different time horizons. While I agree that incentives are the best way to move population-level behavior, and is usually a good starting point for understanding individual behavior, your categorical statement doesn't have those caveats.
- andi999 4y ago
  Billion dollar question is what a better proxy would be.
  - mike_hearn 4y ago
    Possibly a reformed patent system, e.g. one in which time limits are variable or in which you don't get a full monopoly (forced licensing terms). At one point I was researching an alternative patent system in which you don't get a monopoly on the idea, but rather for every next licensee they have to pay in a share of the total value of the patent such that e.g. the first licensee pays half which goes direct to the original filer, the next licensee pays a third and that money is split between the other two licensees, the next pays a quarter etc. So the patent gets cheaper over time to license, but you lose the first mover advantages. The problem with that system is that the "value" would have to be determined in some independent manner, e.g. an audited cost+ basis, which wouldn't be reflective of the true value of the idea. Granting time-limited monopoly rights allows the idea to be priced via normal market mechanisms, at the cost of high transaction costs. At any rate, you get the idea. The justification for academic research is that the private sector, supposedly, won't do long term basic science and that grant recipients will. That's a very dubious set of assumptions. We see basic science being done by private labs all over the place, most obviously in AI but also in many other areas of computer science, we see it in the biotech sector, we see it in semiconductor physics etc. The areas where there's no private investment tend also to be the most controversial areas of academia where many people have concluded that the results are systematically useless e.g. social sciences. And meanwhile what we see amongst grant recipients is systematically unscientific behaviour like knowingly publishing claims that aren't actually true, refusing to share data even after promising to do so, accepting unvalidated modelling as 'science' and a million other things.
jlduan 4y ago
This doesn't make sense. Being scooped means other people published the same idea before you. If he truly believes the released data were fake, he can for sure publish his results. It’s not unusual for scientists to have different views on certain subjects. Some journals even have specific discussion sections for contradictory results. Finding the truth and correcting scientific records is also an achievement. And at least, in the filed of biology/medicine, there are journals accepting “second first” papers, for example https://elifesciences.org/articles/30076 https://elifesciences.org/articles/30076 . And quit a lot of publications are not grand breaking ideas. Incremental improvements can also be published. I don’t understand your friend’s rationale to abandon the whole project he already started and didn’t want to find the truth and prove his theory (maybe it’s a different field thing).
- IanCal 4y ago
  > If he truly believes the released data were fake, he can for sure publish his results. You need the funding to get to that point though, and since it wasn't "new" they weren't able to get funded to research it. That's the meaning I got from the comment.
- Hermitian909 4y ago
  > I don’t understand your friend’s rationale to abandon the whole project he already started and didn’t want to find the truth and prove his theory (maybe it’s a different field thing). He has lots of ideas he wants to try, and he ranks them. The incentives for following this particular idea were reduced by the actions of the other lab, so it dropped in the rankings and he pursued one of the more highly ranked ones.