3 ms·

> There is a countless amount of data, settings, computer code, etc, that is not included with a scientific paper. And that, in itself, is a problem. How can o

> There is a countless amount of data, settings, computer code, etc, that is not included with a scientific paper. And that, in itself, is a problem. How can others identify flaws in the paper if they can't see the data you used? Open Source software includes all the code you need, not just the final binary executable. Any proposed "Github for Science" needs to do the same. And any arguments that "the data is expensive to collect and proprietary" can be countered with "code is expensive to write and proprietary, yet open source still works".

Dayshine 8y ago
Some data cannot be shared due to patient confidentially. In fact, almost all medical trials or epidemiological studies use data that can't be shared outside the study.
- marcus_holmes 8y ago
  surely there are methods of concealing patient identity? We do this with lots of other confidential personal information...
  - Dayshine 8y ago
    But then the research isn't reproducible...
    - stevemadere 8y ago
      Hashing identity solves this. All of the data can be anonymized while retaining the linkage of measurements and procedures performed on the same subject. e.g. We don't know who this guy is, but we do know that every measurement of him was linked to this same enormous random string.
      - Dayshine 8y ago
        Only if your data is very limited in depth. When doing data linking I can usually identify one person in a dataset of almost 1 million from three innocent seeming pieces of data. Pseudonimisation doesn't really work.
joppy 8y ago
And how many people are going to look through all the raw data you used and try to reproduce your results from it? And it's not even possible from that data to verify that the experimental setup that was producing the data in the first place was valid and and well-executed. There are famous examples of widely-used open-source software having bad bugs (for example, OpenSSL) which no-one found, because although it was so widely used, no-one was performing careful enough analysis of the codebase. And programmers are actually paid to look at code all day. No-one working in a research institution is paid or recognised for trying to verify someone else's data or check that their research makes sense - this was the job of peer review. (Even peer review is unpaid and unrecognised at many institutions.)
- marcus_holmes 8y ago
  open source is not a sure-fire way of catching all bugs, but it's still better than closed-source. We're talking an improved situation, not a perfect situation.
overkalix 8y ago
And any arguments that "the data is expensive to collect and proprietary" can be countered with "code is expensive to write and proprietary, yet open source still works". These two are absolutely not equivalent. I'm all for open science, but the comparison with open source code is absurd. With the former you have to invest actual money: infraestructure, materials, suppliers, etc. and of course you have to deal/coordinate those elements. OS is... just time. There is a cost, but it is just an opportunity cost.
- marcus_holmes 8y ago
  Most large Open Source projects are maintained by paid developers, employed at considerable cost by companies that benefit from their time. This is an "actual money" investment of considerable size.

Dayshine 8y ago

Some data cannot be shared due to patient confidentially. In fact, almost all medical trials or epidemiological studies use data that can't be shared outside the study.

marcus_holmes 8y ago
surely there are methods of concealing patient identity? We do this with lots of other confidential personal information...
- Dayshine 8y ago
  But then the research isn't reproducible...
  - stevemadere 8y ago
    Hashing identity solves this. All of the data can be anonymized while retaining the linkage of measurements and procedures performed on the same subject. e.g. We don't know who this guy is, but we do know that every measurement of him was linked to this same enormous random string.
    - Dayshine 8y ago
      Only if your data is very limited in depth. When doing data linking I can usually identify one person in a dataset of almost 1 million from three innocent seeming pieces of data. Pseudonimisation doesn't really work.

joppy 8y ago

And how many people are going to look through all the raw data you used and try to reproduce your results from it? And it's not even possible from that data to verify that the experimental setup that was producing the data in the first place was valid and and well-executed. There are famous examples of widely-used open-source software having bad bugs (for example, OpenSSL) which no-one found, because although it was so widely used, no-one was performing careful enough analysis of the codebase. And programmers are actually paid to look at code all day. No-one working in a research institution is paid or recognised for trying to verify someone else's data or check that their research makes sense - this was the job of peer review. (Even peer review is unpaid and unrecognised at many institutions.)

marcus_holmes 8y ago
open source is not a sure-fire way of catching all bugs, but it's still better than closed-source. We're talking an improved situation, not a perfect situation.

overkalix 8y ago

And any arguments that "the data is expensive to collect and proprietary" can be countered with "code is expensive to write and proprietary, yet open source still works". These two are absolutely not equivalent. I'm all for open science, but the comparison with open source code is absurd. With the former you have to invest actual money: infraestructure, materials, suppliers, etc. and of course you have to deal/coordinate those elements. OS is... just time. There is a cost, but it is just an opportunity cost.

marcus_holmes 8y ago
Most large Open Source projects are maintained by paid developers, employed at considerable cost by companies that benefit from their time. This is an "actual money" investment of considerable size.