3 ms·

>Uploading your code to Github isn't a solution Why not? This sounds like exactly the solution. It's no burden on the researcher - they don't have to alter the

>Uploading your code to Github isn't a solution Why not? This sounds like exactly the solution. It's no burden on the researcher - they don't have to alter their research methods to fit a new system, they just dump the code, and leave it up to other users to reimplement. Building web tools to allow research sounds like precisely the wrong solution, at least in the short to medium term. Research funding doesn't go far enough as it is, you're not going to get researchers changing their processes entirely for no gain. What if they need custom hardware, or access to tools and libraries that haven't been implemented? Sharing _is_ default and easy. That's why Github has exploded.

robotresearcher 10y ago
You forgot to consider IP issues. Schools vary a lot on their policies by but for many the code belongs to the school and can not be open sourced without permission which requires extra work. Funding sources have their own IP deals to consider too. It all sounds so easy until you look at the actual constraints. Professors are usually smart and experienced and they have thought about this stuff a lot. If it was as easy as you thought, it wouldn't be a problem. I publish all code as a matter of lab policy. I chose where to set up my lab partly so that I was able to do this. Not everyone has this luxury or makes this a priority.
- wyattpeak 10y ago
  You're right, I did, but I wasn't actually addressing ease in some absolute sense, I was talking about the appropriate tools for the job. If researchers have the legal right to publish their work, I can't see any reason why github wouldn't be exactly the place they'd share it, rather than some custom online research system as proposed by the parent. That said, I don't have any experience in CS research, it's not my field, so I may be wrong about that, do tell if so.
  - robotresearcher 10y ago
    We share our code on github and publish the commit hash for the code that generated the results in the paper. That way we can continue to develop the code after publication but readers can retrieve the exact code described in the paper if they wish. Simple and effective. But again, the IP rules at my university allow this at my sole discretion, which is unusual.
PeterisP 10y ago
We manage our code on a private github repository, but sharing it would be impossible - the research code relies on various code pieces developed during 5+ different projects with varying IP policies, and we can't make it available to public without either getting permission from all of the relevant organizations and companies (not gonna happen) or getting specific targeted funding to replace/rewrite those components to enable the whole thing to be public - and that's not gonna happen either. Political influence from the top (non-scientific top - the funding sources mostly) is the only way that can improve anything for this problem, the scientists don't really have any realistic way to do that - i.e. one that doesn't expect to sacrifice their scientific and personal goals to go significantly against the current incentive system just to slightly improve openness in their fields. The same goes about datasets - quite a few of the more interesting datasets from industry to do science on are available only under very harsh conditions. You can do very interesting and useful analysis that cannot be reproduced by anyone else, because they are unlikely to ever get access to that particular set of data. An idealistic grad student working on a greenfield project could start and keep that project as open and reproducible as would be best - but most of them work on basis of existing projects with lots of legacy, and there it's different.

robotresearcher 10y ago

You forgot to consider IP issues. Schools vary a lot on their policies by but for many the code belongs to the school and can not be open sourced without permission which requires extra work. Funding sources have their own IP deals to consider too. It all sounds so easy until you look at the actual constraints. Professors are usually smart and experienced and they have thought about this stuff a lot. If it was as easy as you thought, it wouldn't be a problem. I publish all code as a matter of lab policy. I chose where to set up my lab partly so that I was able to do this. Not everyone has this luxury or makes this a priority.

wyattpeak 10y ago
You're right, I did, but I wasn't actually addressing ease in some absolute sense, I was talking about the appropriate tools for the job. If researchers have the legal right to publish their work, I can't see any reason why github wouldn't be exactly the place they'd share it, rather than some custom online research system as proposed by the parent. That said, I don't have any experience in CS research, it's not my field, so I may be wrong about that, do tell if so.
- robotresearcher 10y ago
  We share our code on github and publish the commit hash for the code that generated the results in the paper. That way we can continue to develop the code after publication but readers can retrieve the exact code described in the paper if they wish. Simple and effective. But again, the IP rules at my university allow this at my sole discretion, which is unusual.

PeterisP 10y ago

We manage our code on a private github repository, but sharing it would be impossible - the research code relies on various code pieces developed during 5+ different projects with varying IP policies, and we can't make it available to public without either getting permission from all of the relevant organizations and companies (not gonna happen) or getting specific targeted funding to replace/rewrite those components to enable the whole thing to be public - and that's not gonna happen either. Political influence from the top (non-scientific top - the funding sources mostly) is the only way that can improve anything for this problem, the scientists don't really have any realistic way to do that - i.e. one that doesn't expect to sacrifice their scientific and personal goals to go significantly against the current incentive system just to slightly improve openness in their fields. The same goes about datasets - quite a few of the more interesting datasets from industry to do science on are available only under very harsh conditions. You can do very interesting and useful analysis that cannot be reproduced by anyone else, because they are unlikely to ever get access to that particular set of data. An idealistic grad student working on a greenfield project could start and keep that project as open and reproducible as would be best - but most of them work on basis of existing projects with lots of legacy, and there it's different.