4 ms·

So, $1 of public funding triggers public code release? Or is there a threshold? No one has been able to tell me why the need for reproducibility requires softw

by dalke 10y ago

So, $1 of public funding triggers public code release? Or is there a threshold? No one has been able to tell me why the need for reproducibility requires software freedom. Consider the program 'nauty'. It is available in source code form for anyone to review, but it cannot be used for military purposes. That's not free, certainly. But isn't that enough to call it good science? Similarly, consider the clause "only for use in verifying the result of paper X". That's also not free. But it serves the goal of letting others be able to verify X. Also, you haven't gone far enough. It's not only the license that matters, but access. You have to mandate that either anyone can get access to the code for no/low cost for some years (since I can sell my GPL'ed software for $30,000 or take down the download link once published), or link publication with a required submission to some repository with the mission of keeping all that source and data around, available to anyone, at no cost.

eru 10y ago
> So, $1 of public funding triggers public code release? Or is there a threshold? Yes, 1$, no threshold necessary. For projects where the cost of publication of data is not worth the dollar, people will just not accept the dollar. (That's where a 'natural threshold' comes in.)
- dalke 10y ago
  So, if I have a summer intern who is paid by the government, and helps on my otherwise non-government-funded project, then is that enough to trigger the requirement? Or, if I use a government facility, like a supercomputing center, then that also trigger the release requirement? Even if no money changes hands? What about a government network? It sounds very tricky. If I get a government grant to work on a project, and that grant buys equipment X, which I also use for another project, then must both projects be subject to this form of release? Even if the second project is 10 years later? If in working on project X, but in the process of working on X I find some new knowledge Y, and publish it, even though that has nothing to do with the grant from the original X, does that count? (Think AT&T's observation of the cosmic microwave background, when their goal was to reduce noise in terrestrial microwave communications.) It seems like a very complicated scheme.
  - oldmanjay 10y ago
    No it's very simple. If you want "the people" to pay for you to do things at all, you owe them all of the things or you don't get their money. Your objections make it seem complicated but this feels like an effort to muddy fairly clear waters.
    - dalke 10y ago
      I'm pointing out that it's impossible to work this way. People don't like it. The accounting systems aren't set up for this. The entire post-war research system isn't structured this way.
      - jerf 10y ago
        The entire post-war research system is set up to create endless streams of non-replicable studies based on incorrectly-applied statistics all the while convincing itself that it's doing something useful while wasting billions upon billions of dollars. I'm not to impressed by "well, that's not how it works right now". The whole problem is "how it works right now". That's what we're discussing, the need for it to not work that way.
        dalke 10y ago
        You want to change the system. I understand that. We have many systems to go on over the last few hundred years of science. We have the pre-war system, primarily funded by private philanthropy. We have the communist system. None of them seem to create the stream of highly replicable studies you want. That may indicate something deep about how people work and how science is really done, and suggest that your admirable goals are not tenable.
        jerf 10y ago
        I tend to agree, actually. In some sense the real solution is take science off its pedestal, as it does not generally deserve to be up there. The 17th-19th centuries were in some sense a fluke of low-hanging fruit, and the science of most of the 20th and the 21st centuries do not deserve to be regarded with the same worshipful gaze, a word I choose carefully. By taking it off its pedestal and subjecting it to a lot more scrutiny, we'd all win, including science in general. This is not necessarily because we're worse people than them, but because the problem is now much much harder. It's always better to acknowledge that hard problems are hard, rather than trying to solve them by pretending they're easy. As for the model I would propose, I believe all funding models are fundamentally flawed, and the best model is all of them at once, so hopefully the flaws at least sort of cancel out. At the moment, that generally means seeking a decrease of the current government funding strategy and breaking the peer review monopolies, not because either of them are necessarily especially bad, but because they are too powerful and their flaws are coming to define the flaws of science in general. Some of this would just be a mindset change, to recognize that "research" isn't isomorphic to "producing peer-reviewed papers" and that there's nothing wrong with setting up some equivalents of Xerox Park in other disciplines. Potentially with government money, since my point is more about multiple models than the literal funding sources. If "science" as it is practiced today was less pedestalized, this would be a much less horrifying suggestion.

by dalke 10y ago

eru 10y ago

> So, $1 of public funding triggers public code release? Or is there a threshold? Yes, 1$, no threshold necessary. For projects where the cost of publication of data is not worth the dollar, people will just not accept the dollar. (That's where a 'natural threshold' comes in.)

dalke 10y ago
So, if I have a summer intern who is paid by the government, and helps on my otherwise non-government-funded project, then is that enough to trigger the requirement? Or, if I use a government facility, like a supercomputing center, then that also trigger the release requirement? Even if no money changes hands? What about a government network? It sounds very tricky. If I get a government grant to work on a project, and that grant buys equipment X, which I also use for another project, then must both projects be subject to this form of release? Even if the second project is 10 years later? If in working on project X, but in the process of working on X I find some new knowledge Y, and publish it, even though that has nothing to do with the grant from the original X, does that count? (Think AT&T's observation of the cosmic microwave background, when their goal was to reduce noise in terrestrial microwave communications.) It seems like a very complicated scheme.
- oldmanjay 10y ago
  No it's very simple. If you want "the people" to pay for you to do things at all, you owe them all of the things or you don't get their money. Your objections make it seem complicated but this feels like an effort to muddy fairly clear waters.
  - dalke 10y ago
    I'm pointing out that it's impossible to work this way. People don't like it. The accounting systems aren't set up for this. The entire post-war research system isn't structured this way.
    - jerf 10y ago
      The entire post-war research system is set up to create endless streams of non-replicable studies based on incorrectly-applied statistics all the while convincing itself that it's doing something useful while wasting billions upon billions of dollars. I'm not to impressed by "well, that's not how it works right now". The whole problem is "how it works right now". That's what we're discussing, the need for it to not work that way.
      - dalke 10y ago
        You want to change the system. I understand that. We have many systems to go on over the last few hundred years of science. We have the pre-war system, primarily funded by private philanthropy. We have the communist system. None of them seem to create the stream of highly replicable studies you want. That may indicate something deep about how people work and how science is really done, and suggest that your admirable goals are not tenable.
        jerf 10y ago
        I tend to agree, actually. In some sense the real solution is take science off its pedestal, as it does not generally deserve to be up there. The 17th-19th centuries were in some sense a fluke of low-hanging fruit, and the science of most of the 20th and the 21st centuries do not deserve to be regarded with the same worshipful gaze, a word I choose carefully. By taking it off its pedestal and subjecting it to a lot more scrutiny, we'd all win, including science in general. This is not necessarily because we're worse people than them, but because the problem is now much much harder. It's always better to acknowledge that hard problems are hard, rather than trying to solve them by pretending they're easy. As for the model I would propose, I believe all funding models are fundamentally flawed, and the best model is all of them at once, so hopefully the flaws at least sort of cancel out. At the moment, that generally means seeking a decrease of the current government funding strategy and breaking the peer review monopolies, not because either of them are necessarily especially bad, but because they are too powerful and their flaws are coming to define the flaws of science in general. Some of this would just be a mindset change, to recognize that "research" isn't isomorphic to "producing peer-reviewed papers" and that there's nothing wrong with setting up some equivalents of Xerox Park in other disciplines. Potentially with government money, since my point is more about multiple models than the literal funding sources. If "science" as it is practiced today was less pedestalized, this would be a much less horrifying suggestion.