4 ms·

I like what you're saying. Do you (or anyone on this thread) have any comments/thoughts on the following? During a conversation with an academic researcher (no

I like what you're saying. Do you (or anyone on this thread) have any comments/thoughts on the following? During a conversation with an academic researcher (non-Computer Science) friend, when I brought up the topic of data sharing, especially in context of the infamous "replication crisis", they have their reasons not sharing. I'm loosely paraphrasing here, while trying hard not to misrepresent/misremember their exact views: "I want to protect my data; I don't have enough time to present my data in a presentable form; and more importantly, they'll just steal my idea and go present it as theirs—and I might lose funding" ... and so on. I can empathize with the academic pressure of "publish or perish". And not least of all, "need some food on the table, and roof over my head". But I still wonder, there must be other effective ways to gently persuade a said researcher (especially in the 'soft sciences'—I'm not using the term derogatorily) on the importance of sharing data that allows reproducibility of a given experiment?

theptip 6y ago
One angle that I think is worth exploring is the funding bodies putting requirements in place around publishing data (and the quality of that data) as a condition around funding. If NIST required that you publish the data (say within N years to cover the concern about getting scooped on follow-up papers), and dinged you on future funding applications if you didn't meet their quality/reproducibility metrics, perhaps that would help to align incentives. This is the same sort of idea as requiring research from public funding to be put in open-access journals so that the public can benefit from it.
- kashyapc 6y ago
  Both your points sound promising to me: requirements by funding bodies and time limit for publishing data. But since I'm an academic layman, I can't punch any valid holes in the idea. :-) Regardless, I hope this idea gets discussed more by the real stake holders. Perhaps you might to do a public write-up of the idea.
kerkeslager 6y ago
To be honest, I don't think this is going to be solved from the bottom up. I think a lot of scientists know that they are making compromises between doing science and pursuing their career. But we can't reasonably ask people to do better science when better science means living on an adjunct salary for the rest of their life. The change has to come from publishers. The replication crisis will continue until publishers incentivize replication.
- kashyapc 6y ago
  One researcher-friend did admit of the compromise you speak of, and the internal struggle that goes with it. What remains is the "small matter" of persuading the publishers to action.
  - kerkeslager 6y ago
    That may work with some publishers--MIT Press in particular seems particularly amenable to change, probably because their funding isn't dependent on the popularity of their publications. But the profit-motivated publishers who have a "working" business model aren't going to be persuaded to change it easily. We've watched hundreds of brick and mortar video stores and newspapers go out of business rather than step into the future--if history is any teacher here, the Elseviers of the world will go bankrupt rather than change. And that's a good thing. A revolution in academic science led by Elsevier would be like King George leading the American Revolution. Real fundamental change isn't going to come from the profiteers who caused the problem in the first place.
thedudeabides5 6y ago
It's just too damn hard to share data, even outputs, let alone the horrific excel or python files necessary to clean, model and visualize the results. Finance has the exact same problem.

theptip 6y ago

One angle that I think is worth exploring is the funding bodies putting requirements in place around publishing data (and the quality of that data) as a condition around funding. If NIST required that you publish the data (say within N years to cover the concern about getting scooped on follow-up papers), and dinged you on future funding applications if you didn't meet their quality/reproducibility metrics, perhaps that would help to align incentives. This is the same sort of idea as requiring research from public funding to be put in open-access journals so that the public can benefit from it.

kashyapc 6y ago
Both your points sound promising to me: requirements by funding bodies and time limit for publishing data. But since I'm an academic layman, I can't punch any valid holes in the idea. :-) Regardless, I hope this idea gets discussed more by the real stake holders. Perhaps you might to do a public write-up of the idea.

kerkeslager 6y ago

To be honest, I don't think this is going to be solved from the bottom up. I think a lot of scientists know that they are making compromises between doing science and pursuing their career. But we can't reasonably ask people to do better science when better science means living on an adjunct salary for the rest of their life. The change has to come from publishers. The replication crisis will continue until publishers incentivize replication.

kashyapc 6y ago
One researcher-friend did admit of the compromise you speak of, and the internal struggle that goes with it. What remains is the "small matter" of persuading the publishers to action.
- kerkeslager 6y ago
  That may work with some publishers--MIT Press in particular seems particularly amenable to change, probably because their funding isn't dependent on the popularity of their publications. But the profit-motivated publishers who have a "working" business model aren't going to be persuaded to change it easily. We've watched hundreds of brick and mortar video stores and newspapers go out of business rather than step into the future--if history is any teacher here, the Elseviers of the world will go bankrupt rather than change. And that's a good thing. A revolution in academic science led by Elsevier would be like King George leading the American Revolution. Real fundamental change isn't going to come from the profiteers who caused the problem in the first place.

thedudeabides5 6y ago

It's just too damn hard to share data, even outputs, let alone the horrific excel or python files necessary to clean, model and visualize the results. Finance has the exact same problem.