5 ms·

I’m curious: could anyone knowledgeable share a bit about these QCD lattice models — are they incredibly complicated? Are there open source implementations of t

I’m curious: could anyone knowledgeable share a bit about these QCD lattice models — are they incredibly complicated? Are there open source implementations of these models and if so, would the code make any sense to a non-physicist programmer?

lisper 5y ago
> are they incredibly complicated? That depends on what you mean by "complicated". Conceptually they are not too bad, it's basically just quantum field theory (which some might consider "incredibly complicated" so YMMV). But the devil is in the details. From the article: "While physicists know the exact equation that defines the strong force — the fundamental force that binds quarks together to make the protons and neutrons in the hearts of atoms, as well as other composite particles like tetraquarks — they can rarely solve this strange, endlessly iterative equation, so they struggle to predict the strong force’s effects." > Are there open source implementations of these models That I don't know. > and if so, would the code make any sense to a non-physicist programmer? Having seen code written by scientists, I can confidently answer this with: almost certainly not.
- datameta 5y ago
  >Having seen code written by scientists, I can confidently answer this with: almost certainly not. Got a kick out of that, thanks. I think there is a kind of compartmentalization of environments going on when one engages in scientific programming. It seems to often be structured more akin to math than code. I once worked with a brilliant multitalented individual who held a PhD in a specific field of physics that I cannot recall. In one instance they ported the orchestration software for a supercomputer from python into java, in a week. It was beautiful to read. This same person used single letter variables in their ML models and had virtually zero comments which made it somewhat difficult for me to follow their updates after a week of having not seen the code.
  - ethbr0 5y ago
    To me, it's a difference in professional gestalts(?) in highly trained individuals. Physical engineers, especially, have a very formalized, standardized way of looking at the world. They all share it, and so their code follows from that. Physicists, biologists, software developers all have different ways of looking at the world (some more standardized, some less). Our code follows from that. So while we may gripe they don't leverage (insert language arcana / convention), my intuition is our code would still look very different even if they did. Biggest example, as you point out: math-structured code, from math-heavy disciplines.
  - lisper 5y ago
    Here's an analogy that I can speak from more direct experience: if you study the theory of elliptic curves, and then look at elliptic curve cryptography code, there will appear to be no connection between the two. This is because in between the theory and the code are a zillion implementation details and tricks of the trade that aren't generally mentioned in the theory. Add to that the fact that the people who write scientific code aren't coders by trade, so they learn to write code that is just good enough but no better, and the result is something that looks like a horrible mess. (Also, the code is often written by grad students, so that makes it even worse. I look back on some of the code that I wrote back when I was a grad student that was the basis for publications and I shudder.)
  - tsimionescu 5y ago
    This reminded me of the chemical PhD who created Clasp, a full common lisp implementation using LLVM as a JIT compiler, able to call even template C++ code from Common Lisp (a feat which no other language I know of even tries, except perhaps .NET with managed C++). [0] https://github.com/clasp-developers/clasp https://github.com/clasp-developers/clasp
  - NineStarPoint 5y ago
    From what I’ve heard the core of the issue is it’s similar to non-online game development. Most scientists make some programs for a single paper, and then just start from scratch for a new project. When you don’t have to care about long term maintainability readability becomes much less of a concern to people.
- andi999 5y ago
  The Fortran code at particle research groups I saw looked pretty solid to me.
  - lisper 5y ago
    "Fortran" and "solid" are not two words that go together in my mind ;-) But the OP didn't ask if the code was good (or "solid"), they asked if it would make any sense to a non-physicist programmer. And the answer to that question, I'm guessing, is most likely not. But I would be very happy if I turned out to be wrong about that.
    - andi999 5y ago
      It probably boils down to if there are any non physicist Fortran programmers...
    - TheRealKing 5y ago
      You can write bad code in any language. The verbosity and restrictions of (modern) Fortran standards leave significantly less room for developer mistakes, much better than C/C++.
jerf 5y ago
I'm not an expert on particle physics, but the verbiage sounds to me like one of the standard problems in physics: We can write the differential equations. But even simple differential equations can be unsolvable. For example, consider the simple Three Body Problem. The differential equations are simple enough that you can use them as an introduction to the concept of differential equations themselves, but the Three Body Problem is not in general solvable. The three body problem can generally be acceptably approximated for a reasonable period of time. But that problem only involves inverse squaring of distances. Strong forces decay much faster than that, which makes them more sensitive to errors. Plus you end up with one of the fundamental problems we have trying to understand our universe, which is just how monstrously enormous and monstrously slow we humans are. We operate at "meter" scales and "second" time frames, and particle physics operates at somewhere around 30 and 40 orders of magnitude smaller, respectively. (Not quite all the way down to the Planck sizes, but closer to those than to the macroscopic world.) So when you try to numerically approximate the differential equations, you don't get very far in time or space before your approximations have critically diverged from reality. It's like we're trying to work out the fundamentals of chemistry and our primary tool is smashing planets together.
- klodolph 5y ago
  When people say that the three-body problem is "unsolvable" they really mean that there's not a way to write out an analytic solution to the problem. In the same way, you can call quintic polynomials "unsolvable", because there's no way to take arbitrary quintic polynomial equations and express the solutions as ordinary algebraic formulas. However, it's very misleading to call quintic equations unsolvable, because we know where the solutions are, and we can use various numeric methods to calculate the solution with arbitrary precision. Any time we can calculate the answer with as much precision as we want, I'd like to say that the problem is "solved" in a very real and meaningful way. The problem is worse with quantum mechanics. With quantum mechanics, not only do we lack analytic solutions to many of the equations used in QM, but we also lack good numeric solutions (using real hardware, at least).
  - ethbr0 5y ago
rrss 5y ago
> Are there open source implementations of these models there are some open source packages for lattice QCD. e.g. https://jeffersonlab.github.io/chroma/ https://jeffersonlab.github.io/chroma/
- dmbaggett 5y ago
  Also found this GitHub topic page: https://github.com/topics/lattice-qcd https://github.com/topics/lattice-qcd
- zbyszek 5y ago
  Goodness me, that's a blast from my very distant past! The venerable MILC code is still available: https://web.physics.utah.edu/~detar/milc/milc_qcd.html https://web.physics.utah.edu/~detar/milc/milc_qcd.html
maxnoe 5y ago
A Quick search found this: https://github.com/akio-tomiya/LatticeQCD.jl https://github.com/akio-tomiya/LatticeQCD.jl Not my field of expertise though, I am in experimental gamma-ray astronomy
ahefner 5y ago
I've wondered this too, and if it's possible for a toy implementation to help teach the concepts, but I suspect not. Probably the translation to code obscures more than it illustrates.
- evanb 5y ago
  You might enjoy Lepage’s “Lattice QCD for novices” which develops a lot of the ideas without getting bogged down in the specifics of QCD (it focuses on the harmonic oscillator + a harmonic oscillator). [0] https://arxiv.org/abs/hep-lat/0506036 https://arxiv.org/abs/hep-lat/0506036
evanb 5y ago
I’m a LQCD practitioner. There are a variety of open-source implementations; I’ll just point to a few. The one funded by the Department of Energy through the SciDAC program is the USQCD software stack [0]. There’s also the GPU library quda, which is maintained by Nvidia employees (and others in the community). There’s Grid [2], development led by Edinburgh in close collaboration with intel (to make sure it compiles down to sensible high-performance primatives). There’s openQCD [3], coordinated by CERN researchers. As to whether it’ll be readable to you—-maybe? How transparent each library is differs. The most important parts are typically (1) the generation of gauge configurations (typically by HMC, which was discovered by the LQCD community [4]), which are MCMC samples and (2) the calculation of observable on each sample. Both rely on highly optimized (and preconditioned, and maybe multigrid-ed) linear solves—-the most important kernel. Some libraries are written to be as transparent as possible; some to be as portable as possible. All are written to handle massive data parallelism across hundreds of high-performance nodes with some mixture of OpenMP, MPI, #pragma acceleration, etc. Finally, the code will only “make (big picture) sense” to you if you understand lattice quantum field theory. [0] http://usqcd-software.github.io/ http://usqcd-software.github.io/ [1] http://lattice.github.io/quda/ http://lattice.github.io/quda/ [2] https://github.com/paboyle/Grid https://github.com/paboyle/Grid [3] https://luscher.web.cern.ch/luscher/openQCD/ https://luscher.web.cern.ch/luscher/openQCD/ [4] https://www.sciencedirect.com/science/article/abs/pii/037026938791197X https://www.sciencedirect.com/science/article/abs/pii/037026...
zbyszek 5y ago
As far as I recall, QCD gives you a nice simple formula - a path integral over quantum operators - for any quantity you want. Lattice QCD works out the answer with great big Monte-Carlo integration. The tricky bit is defining a field theory on a finite discrete space-time lattice that is provably equivalent to the continuum theory after controlled extrapolations to the continuum. And you have to show that you can define the correct operators in the lattice theory that really do give the quantities you want. Then you need to implement this in code that runs blazingly fast over huge data sets. These considerations, and the fact that there are all sorts of solutions to these problems which might supported by a code base, tend to make code implementations complicated.

lisper 5y ago

> are they incredibly complicated? That depends on what you mean by "complicated". Conceptually they are not too bad, it's basically just quantum field theory (which some might consider "incredibly complicated" so YMMV). But the devil is in the details. From the article: "While physicists know the exact equation that defines the strong force — the fundamental force that binds quarks together to make the protons and neutrons in the hearts of atoms, as well as other composite particles like tetraquarks — they can rarely solve this strange, endlessly iterative equation, so they struggle to predict the strong force’s effects." > Are there open source implementations of these models That I don't know. > and if so, would the code make any sense to a non-physicist programmer? Having seen code written by scientists, I can confidently answer this with: almost certainly not.

datameta 5y ago
>Having seen code written by scientists, I can confidently answer this with: almost certainly not. Got a kick out of that, thanks. I think there is a kind of compartmentalization of environments going on when one engages in scientific programming. It seems to often be structured more akin to math than code. I once worked with a brilliant multitalented individual who held a PhD in a specific field of physics that I cannot recall. In one instance they ported the orchestration software for a supercomputer from python into java, in a week. It was beautiful to read. This same person used single letter variables in their ML models and had virtually zero comments which made it somewhat difficult for me to follow their updates after a week of having not seen the code.
- ethbr0 5y ago
  To me, it's a difference in professional gestalts(?) in highly trained individuals. Physical engineers, especially, have a very formalized, standardized way of looking at the world. They all share it, and so their code follows from that. Physicists, biologists, software developers all have different ways of looking at the world (some more standardized, some less). Our code follows from that. So while we may gripe they don't leverage (insert language arcana / convention), my intuition is our code would still look very different even if they did. Biggest example, as you point out: math-structured code, from math-heavy disciplines.
- lisper 5y ago
  Here's an analogy that I can speak from more direct experience: if you study the theory of elliptic curves, and then look at elliptic curve cryptography code, there will appear to be no connection between the two. This is because in between the theory and the code are a zillion implementation details and tricks of the trade that aren't generally mentioned in the theory. Add to that the fact that the people who write scientific code aren't coders by trade, so they learn to write code that is just good enough but no better, and the result is something that looks like a horrible mess. (Also, the code is often written by grad students, so that makes it even worse. I look back on some of the code that I wrote back when I was a grad student that was the basis for publications and I shudder.)
- tsimionescu 5y ago
  This reminded me of the chemical PhD who created Clasp, a full common lisp implementation using LLVM as a JIT compiler, able to call even template C++ code from Common Lisp (a feat which no other language I know of even tries, except perhaps .NET with managed C++). [0] https://github.com/clasp-developers/clasp https://github.com/clasp-developers/clasp
- NineStarPoint 5y ago
  From what I’ve heard the core of the issue is it’s similar to non-online game development. Most scientists make some programs for a single paper, and then just start from scratch for a new project. When you don’t have to care about long term maintainability readability becomes much less of a concern to people.
andi999 5y ago
The Fortran code at particle research groups I saw looked pretty solid to me.
- lisper 5y ago
  "Fortran" and "solid" are not two words that go together in my mind ;-) But the OP didn't ask if the code was good (or "solid"), they asked if it would make any sense to a non-physicist programmer. And the answer to that question, I'm guessing, is most likely not. But I would be very happy if I turned out to be wrong about that.
  - andi999 5y ago
    It probably boils down to if there are any non physicist Fortran programmers...
  - TheRealKing 5y ago
    You can write bad code in any language. The verbosity and restrictions of (modern) Fortran standards leave significantly less room for developer mistakes, much better than C/C++.

jerf 5y ago

I'm not an expert on particle physics, but the verbiage sounds to me like one of the standard problems in physics: We can write the differential equations. But even simple differential equations can be unsolvable. For example, consider the simple Three Body Problem. The differential equations are simple enough that you can use them as an introduction to the concept of differential equations themselves, but the Three Body Problem is not in general solvable. The three body problem can generally be acceptably approximated for a reasonable period of time. But that problem only involves inverse squaring of distances. Strong forces decay much faster than that, which makes them more sensitive to errors. Plus you end up with one of the fundamental problems we have trying to understand our universe, which is just how monstrously enormous and monstrously slow we humans are. We operate at "meter" scales and "second" time frames, and particle physics operates at somewhere around 30 and 40 orders of magnitude smaller, respectively. (Not quite all the way down to the Planck sizes, but closer to those than to the macroscopic world.) So when you try to numerically approximate the differential equations, you don't get very far in time or space before your approximations have critically diverged from reality. It's like we're trying to work out the fundamentals of chemistry and our primary tool is smashing planets together.

klodolph 5y ago
When people say that the three-body problem is "unsolvable" they really mean that there's not a way to write out an analytic solution to the problem. In the same way, you can call quintic polynomials "unsolvable", because there's no way to take arbitrary quintic polynomial equations and express the solutions as ordinary algebraic formulas. However, it's very misleading to call quintic equations unsolvable, because we know where the solutions are, and we can use various numeric methods to calculate the solution with arbitrary precision. Any time we can calculate the answer with as much precision as we want, I'd like to say that the problem is "solved" in a very real and meaningful way. The problem is worse with quantum mechanics. With quantum mechanics, not only do we lack analytic solutions to many of the equations used in QM, but we also lack good numeric solutions (using real hardware, at least).
- ethbr0 5y ago

rrss 5y ago

> Are there open source implementations of these models there are some open source packages for lattice QCD. e.g. https://jeffersonlab.github.io/chroma/ https://jeffersonlab.github.io/chroma/

dmbaggett 5y ago
Also found this GitHub topic page: https://github.com/topics/lattice-qcd https://github.com/topics/lattice-qcd
zbyszek 5y ago
Goodness me, that's a blast from my very distant past! The venerable MILC code is still available: https://web.physics.utah.edu/~detar/milc/milc_qcd.html https://web.physics.utah.edu/~detar/milc/milc_qcd.html

maxnoe 5y ago

A Quick search found this: https://github.com/akio-tomiya/LatticeQCD.jl https://github.com/akio-tomiya/LatticeQCD.jl Not my field of expertise though, I am in experimental gamma-ray astronomy

ahefner 5y ago

I've wondered this too, and if it's possible for a toy implementation to help teach the concepts, but I suspect not. Probably the translation to code obscures more than it illustrates.

evanb 5y ago
You might enjoy Lepage’s “Lattice QCD for novices” which develops a lot of the ideas without getting bogged down in the specifics of QCD (it focuses on the harmonic oscillator + a harmonic oscillator). [0] https://arxiv.org/abs/hep-lat/0506036 https://arxiv.org/abs/hep-lat/0506036

evanb 5y ago

I’m a LQCD practitioner. There are a variety of open-source implementations; I’ll just point to a few. The one funded by the Department of Energy through the SciDAC program is the USQCD software stack [0]. There’s also the GPU library quda, which is maintained by Nvidia employees (and others in the community). There’s Grid [2], development led by Edinburgh in close collaboration with intel (to make sure it compiles down to sensible high-performance primatives). There’s openQCD [3], coordinated by CERN researchers. As to whether it’ll be readable to you—-maybe? How transparent each library is differs. The most important parts are typically (1) the generation of gauge configurations (typically by HMC, which was discovered by the LQCD community [4]), which are MCMC samples and (2) the calculation of observable on each sample. Both rely on highly optimized (and preconditioned, and maybe multigrid-ed) linear solves—-the most important kernel. Some libraries are written to be as transparent as possible; some to be as portable as possible. All are written to handle massive data parallelism across hundreds of high-performance nodes with some mixture of OpenMP, MPI, #pragma acceleration, etc. Finally, the code will only “make (big picture) sense” to you if you understand lattice quantum field theory. [0] http://usqcd-software.github.io/ http://usqcd-software.github.io/ [1] http://lattice.github.io/quda/ http://lattice.github.io/quda/ [2] https://github.com/paboyle/Grid https://github.com/paboyle/Grid [3] https://luscher.web.cern.ch/luscher/openQCD/ https://luscher.web.cern.ch/luscher/openQCD/ [4] https://www.sciencedirect.com/science/article/abs/pii/037026938791197X https://www.sciencedirect.com/science/article/abs/pii/037026...

zbyszek 5y ago

As far as I recall, QCD gives you a nice simple formula - a path integral over quantum operators - for any quantity you want. Lattice QCD works out the answer with great big Monte-Carlo integration. The tricky bit is defining a field theory on a finite discrete space-time lattice that is provably equivalent to the continuum theory after controlled extrapolations to the continuum. And you have to show that you can define the correct operators in the lattice theory that really do give the quantities you want. Then you need to implement this in code that runs blazingly fast over huge data sets. These considerations, and the fact that there are all sorts of solutions to these problems which might supported by a code base, tend to make code implementations complicated.