4 ms·

> OK, so they don't want to share their source code; that's fine. No, it is not. It is the exact reason why rheir results are not trust worthy. Publish the co

by maxnoe 6y ago

> OK, so they don't want to share their source code; that's fine. No, it is not. It is the exact reason why rheir results are not trust worthy. Publish the code, let it be checked by the peers. Closed source code has no place in science and most journals now rightly demand open code for the publications.

evanb 6y ago
I appreciate the absolutist position. However, everybody agrees on what code _must do_. If you need to solve a (massive) system of linear equations (as happens often in LQCD) you can then take your alleged solution and plug it in and check. A variety of those sorts of things prevent you from doing anything too wrong. If you screw up gauge invariance, for example, you will get 0. There are agreed-upon small examples. Plus other benchmarks---they computed the hadronic spectrum. They computed the splitting between the proton's mass and the neutron's mass. If you spend hundreds or thousands of man hours optimizing, for example, assembler for a communications-intensive highly-parallel linear solve, it's fair to be reluctant to give it away. If you do others will get the glory (publications / funding). Some people do [ eg. this solver library for the BlueGenes https://www2.ph.ed.ac.uk/~paboyle/bagel/Bagel.html https://www2.ph.ed.ac.uk/~paboyle/bagel/Bagel.html ]. Most people are happy to let others do the hard work of building low-level libraries. But they COULD decide to write custom software that'd go faster. If their custom software reproduce results that community-standard libraries produce that's not nothing.
- dguest 6y ago
  I think this clarifies a misunderstanding I had from your original comment. It sounds like the "secret sauce" for this collaboration includes a set of numerical libraries. They would get relatively little funding, few publications ("glory", as you say), and at best be reduced to a citation (if people remember to cite their libraries) if all they did was improve the backbone of lattice QCD with better software. So instead they keep it internal. It's a bit sad that there's so little glory in writing better numerical libraries, but it's a common problem across the sciences (and in the open source community in general) so I can believe they'd be reluctant to share.
  - evanb 6y ago
    > It sounds like the "secret sauce" for this collaboration includes a set of numerical libraries. Indeed. There are really only a limited set of (physics) choices when making these libraries. As long as the discretization you pick goes to QCD in the continuum limit, you can make whatever choices you want. Some choices lead to faster convergence, or easier numerics, or better symmetry, or whatever---at that point it's a cost/benefit analysis. But if your discretization ('lattice action') goes is in the QCD universality class ('has the right continuum limit') you're guaranteed to get the right answer as long as you can extrapolate to the continuum. > It's a bit sad that there's so little glory in writing better numerical libraries. Agreed, but physics departments (by and large) award tenure for doing physics, not for doing computer science. It's hard to get departments to say "yes, your expertise in optimizing GPU code is enough to get you on the tenure track". > It's a common problem across the sciences. [...] I can believe they'd be reluctant to share. The larger community does center around common codes. The biggest players are USQCD http://usqcd-software.github.io/ http://usqcd-software.github.io/ quda http://lattice.github.io/quda/ http://lattice.github.io/quda/ grid https://github.com/paboyle/Grid/ https://github.com/paboyle/Grid/ but there are others, and there are private codes (like BMW's) too. As part of the SciDAC program and now exascale initiative the DOE does fund a few software-focused national lab jobs. But not many.

by maxnoe 6y ago

evanb 6y ago

I appreciate the absolutist position. However, everybody agrees on what code _must do_. If you need to solve a (massive) system of linear equations (as happens often in LQCD) you can then take your alleged solution and plug it in and check. A variety of those sorts of things prevent you from doing anything too wrong. If you screw up gauge invariance, for example, you will get 0. There are agreed-upon small examples. Plus other benchmarks---they computed the hadronic spectrum. They computed the splitting between the proton's mass and the neutron's mass. If you spend hundreds or thousands of man hours optimizing, for example, assembler for a communications-intensive highly-parallel linear solve, it's fair to be reluctant to give it away. If you do others will get the glory (publications / funding). Some people do [ eg. this solver library for the BlueGenes https://www2.ph.ed.ac.uk/~paboyle/bagel/Bagel.html https://www2.ph.ed.ac.uk/~paboyle/bagel/Bagel.html ]. Most people are happy to let others do the hard work of building low-level libraries. But they COULD decide to write custom software that'd go faster. If their custom software reproduce results that community-standard libraries produce that's not nothing.

dguest 6y ago
I think this clarifies a misunderstanding I had from your original comment. It sounds like the "secret sauce" for this collaboration includes a set of numerical libraries. They would get relatively little funding, few publications ("glory", as you say), and at best be reduced to a citation (if people remember to cite their libraries) if all they did was improve the backbone of lattice QCD with better software. So instead they keep it internal. It's a bit sad that there's so little glory in writing better numerical libraries, but it's a common problem across the sciences (and in the open source community in general) so I can believe they'd be reluctant to share.
- evanb 6y ago
  > It sounds like the "secret sauce" for this collaboration includes a set of numerical libraries. Indeed. There are really only a limited set of (physics) choices when making these libraries. As long as the discretization you pick goes to QCD in the continuum limit, you can make whatever choices you want. Some choices lead to faster convergence, or easier numerics, or better symmetry, or whatever---at that point it's a cost/benefit analysis. But if your discretization ('lattice action') goes is in the QCD universality class ('has the right continuum limit') you're guaranteed to get the right answer as long as you can extrapolate to the continuum. > It's a bit sad that there's so little glory in writing better numerical libraries. Agreed, but physics departments (by and large) award tenure for doing physics, not for doing computer science. It's hard to get departments to say "yes, your expertise in optimizing GPU code is enough to get you on the tenure track". > It's a common problem across the sciences. [...] I can believe they'd be reluctant to share. The larger community does center around common codes. The biggest players are USQCD http://usqcd-software.github.io/ http://usqcd-software.github.io/ quda http://lattice.github.io/quda/ http://lattice.github.io/quda/ grid https://github.com/paboyle/Grid/ https://github.com/paboyle/Grid/ but there are others, and there are private codes (like BMW's) too. As part of the SciDAC program and now exascale initiative the DOE does fund a few software-focused national lab jobs. But not many.