4 ms·

I would put the burden of proof on the 'radicals' in this case: if you believe that the traditional scientific research system can be cut in half, and/or replac

I would put the burden of proof on the 'radicals' in this case: if you believe that the traditional scientific research system can be cut in half, and/or replaced by startups and other endeavors, and still produce the same or more advances in science, I'd like to see evidence supporting that hypothesis. Put in Bayesian terms, I put a reasonably strong prior on the current system successfully generating science, so radical changes to it, like significantly gutting universities, need evidence to justify them. If we had metrics that fully explained where scientific advances come from and how they feed back into engineering and other areas, in at least the medium-term, we could use those as a guide for changes. But the metrics so far are about as good as lines-of-code metrics: not completely noise, but relatively low explanatory value. Metrics are only good if they can be validated: if you can come up with metrics for scientific productivity and also show that your metrics explain a large portion of scientific advances and their later use, then they might be worth taking seriously. So far, though, the metrics are relatively poor. But that does not prove the opposite: that scientific advances don't happen, and don't get later commercialized. The fact that one doesn't fully understand why scientific advances happen, and have not successfully found a way to attach lines-of-code-style metrics to measuring them, is not actual evidence that the current system for scientific research doesn't work, or that any particular proposed change will be neutral to positive.

yummyfajitas 13y ago
...if you believe that the traditional scientific research system can be cut in half... Where did you get the idea that I believe this? I'm simply arguing that no endeavor should be exempt from cost/benefit analysis.
- _delirium 13y ago
  If there's a validated metric on the 'benefit' side, sure. What I'm skeptical of is making up some unrigorous numbers to do quantitative cost-benefit analysis when a system isn't well understood enough to be quantified with any degree of confidence in the numbers, at least at present. Assuming anything we can't measure well yet is ~0 is also not a very rigorous approach. One cautious approach could be to try to come up with lower and upper bounds on benefits, e.g. that with high confidence, we can conclude that academic physics research, or academic geography research, or whatever, has had at least $N but not more than $M impact on the U.S. economy, taking into account direct and indirect effects over a period of, say, 30 years. I suspect the space between those bounds would be large, but we would at least be able to say for sure (for certain values of "sure") that quantifiable benefit lies within a certain range. I'd be interested in that research, if there is any, but these kinds of estimates are notoriously difficult to establish, given that history isn't an experimental science.
  - yummyfajitas 13y ago
    Assuming anything we can't measure well yet is ~0 is also not a very rigorous approach. At the end of the day, you need a number - the amount of dollars to fund physics with. This number will have uncertainty attached to it, just like every other number. I agree that such numbers are difficult to establish, they are likely to be strongly dependent on priors, and credible intervals will be large. So what? It's still a better approach than simply throwing arbitrary politically determined amounts of money at endeavors that sound cool to a rationally innumerate and ignorant electorate.
- phaemon 13y ago
  > I'm simply arguing that no endeavor should be exempt from cost/benefit analysis. Why? Is there evidence that cost-benefit analyses actually work?
  - yummyfajitas 13y ago
    No. Similarly, there is no non-circular evidence that statistics works, that double blind clinical trials work, or that any of our basic scientific machinery works. If you adopt a different epistemiology, you will arrive at different conclusions.
    - phaemon 13y ago
      Of course there is! If statistics didn't work, casinos wouldn't work. If our basic scientific machinery didn't work, computers wouldn't work. If cost-benefit analyses didn't work, government projects would often go massively over budget producing nothing, while other areas that were underfunded would occasionally have startling breakthroughs....
      - yummyfajitas 13y ago
        If statistics didn't work, casinos wouldn't work. How do you know this? Oh right, from statistics. Science can't prove your epistemiology correct.
        phaemon 13y ago
        No, I know this because I know a) casinos use statistics and b) casinos actually exist. Look at the desperate efforts you're having to make to try and justify a cost-benefit analysis. You're questioning the validity of tried and tested statistical methods! Why? Shouldn't a CBA be easier to justify if it's so clearly a good idea that should be applied to everything as you stated earlier?
        yummyfajitas 13y ago

yummyfajitas 13y ago

...if you believe that the traditional scientific research system can be cut in half... Where did you get the idea that I believe this? I'm simply arguing that no endeavor should be exempt from cost/benefit analysis.

_delirium 13y ago
If there's a validated metric on the 'benefit' side, sure. What I'm skeptical of is making up some unrigorous numbers to do quantitative cost-benefit analysis when a system isn't well understood enough to be quantified with any degree of confidence in the numbers, at least at present. Assuming anything we can't measure well yet is ~0 is also not a very rigorous approach. One cautious approach could be to try to come up with lower and upper bounds on benefits, e.g. that with high confidence, we can conclude that academic physics research, or academic geography research, or whatever, has had at least $N but not more than $M impact on the U.S. economy, taking into account direct and indirect effects over a period of, say, 30 years. I suspect the space between those bounds would be large, but we would at least be able to say for sure (for certain values of "sure") that quantifiable benefit lies within a certain range. I'd be interested in that research, if there is any, but these kinds of estimates are notoriously difficult to establish, given that history isn't an experimental science.
- yummyfajitas 13y ago
  Assuming anything we can't measure well yet is ~0 is also not a very rigorous approach. At the end of the day, you need a number - the amount of dollars to fund physics with. This number will have uncertainty attached to it, just like every other number. I agree that such numbers are difficult to establish, they are likely to be strongly dependent on priors, and credible intervals will be large. So what? It's still a better approach than simply throwing arbitrary politically determined amounts of money at endeavors that sound cool to a rationally innumerate and ignorant electorate.
phaemon 13y ago
> I'm simply arguing that no endeavor should be exempt from cost/benefit analysis. Why? Is there evidence that cost-benefit analyses actually work?
- yummyfajitas 13y ago
  No. Similarly, there is no non-circular evidence that statistics works, that double blind clinical trials work, or that any of our basic scientific machinery works. If you adopt a different epistemiology, you will arrive at different conclusions.
  - phaemon 13y ago
    Of course there is! If statistics didn't work, casinos wouldn't work. If our basic scientific machinery didn't work, computers wouldn't work. If cost-benefit analyses didn't work, government projects would often go massively over budget producing nothing, while other areas that were underfunded would occasionally have startling breakthroughs....
    - yummyfajitas 13y ago
      If statistics didn't work, casinos wouldn't work. How do you know this? Oh right, from statistics. Science can't prove your epistemiology correct.
      - phaemon 13y ago
        No, I know this because I know a) casinos use statistics and b) casinos actually exist. Look at the desperate efforts you're having to make to try and justify a cost-benefit analysis. You're questioning the validity of tried and tested statistical methods! Why? Shouldn't a CBA be easier to justify if it's so clearly a good idea that should be applied to everything as you stated earlier?
        yummyfajitas 13y ago