6 ms·

Crucial Computer Program for Particle Physics at Risk of Obsolescence

89 points by theafh 4y ago

cratermoon 4y ago
Related discussion on programming and academia: https://news.ycombinator.com/item?id=33671264 https://news.ycombinator.com/item?id=33671264
prof-dr-ir 4y ago
https://github.com/vermaseren/form https://github.com/vermaseren/form
simne 4y ago
This is looking strange, because Particle Physics is one of very few areas, where hardware is not magnitudes cheaper than software. I mean not only computer hardware, which is also non cheap, but budgets for experimental hardware, which not all countries can handle. So, if program is really important, they could rearrange budgets and give some finances for software.
- avidphantasm 4y ago
  Agreed. I would be shocked if an entity like CERN would not be able to support something like FORM out of grant overhead.
moomin 4y ago
It's yet another story of the dumb misalign incentives in academia. Hand the guy a Nobel Prize. See if that moves the needle a bit.
Qem 4y ago
The root cause is academia still didn't adjust the rewards to take into account the computer age. People only get career points by publishing papers, not source code. Software is a fundamental aspect of most current research. We need to recognize researchers that dedicate to write/maintain the required software, awarding the fair amount of career points. Make good software documentation be the equivalent to papers for them, career-wise. It surely would improve the state of academic software. Today students still commonly have to struggle with software stuck in the 70s, usability-wise.
- rowanG077 4y ago
  It would also greatly increase the diffusion into industry of techniques. I often read papers that describe an interesting approach to something I am interested in doing. But often the paper requires me to read many references and spend dozens of hours or more to reach an implementation myself. I simply can't invest that time on something that might work. But if the code were available... then you could quickly test out if something is worth pursuing.
  - Terretta 4y ago
    Arguably, by the time they implemented, they tested, at that point you're just consuming. So another way of handling this to contribute advancement at scale is "exercise left to the reader", and then we readers should publish too. R&D in big tech should contribute as academia does. We have our "Chief Scientists" and "Principal Engineers" titles that recognize, essentially, a tenure track, that should be on the merits of publishing code implementing new techniques.
- Konohamaru 4y ago
  This is a good idea. Imagine when future Nobel prizes are given to physicists who write and maintain software just as much to those who publish papers.
  - Qem 4y ago
    This problem doesn't need any nobels to be solved. Just calibrate the rewards so programmer-scientists don't risk career stagnation if they take the time to develop and maintain tools of critical importance for science.
    - Konohamaru 4y ago
      But imagine if physicists were as recognized for software development as much as for experimental work or theorizing. That's a much more important reward.
- 4y ago
anigbrowl 4y ago
Universities tend to track scientists’ publication records Yet another example of administrators imposing a one-size-fits-all rule and elevating convenience over quality. Every academic I know hates how academia works, but they balk every time I suggest networking with people outside their disciplinary silos to dethrone the administrators. It's like the smartest people in the world are stymied by the idea of a collective action problem that requires working across disciplinary boundaries. For a long time I thought I was missing something fundamental about the complexity of the issue, but after 10 years of asking it seems to literally be that individuals are afraid of getting punished and they don't have any ideas beyond supporting the occasional strike action (that will leave things more or less as they are). I don't get it.
- lowbloodsugar 4y ago
  I mean you basically just described Autism. “Wow, you guys are amazing at your Special Interest and talking with other people with the same Special Interest. Why aren’t you interested at talking with these Neurotypical bean counters about finance stuff?” Not that I’m saying every physics major is Autistic. I myself know several in my graduating class who are not.
  - anigbrowl 4y ago
    I resemble that remark myself. I would only clarify that I think the time for talking with bean counters is long past.
- fnordpiglet 4y ago
  Accountants know the price of everything and the value of nothing.
machina_ex_deus 4y ago
It's pretty remarkable that despite Feynman Diagrams being invented in the 1950s, before computers and computer science, they are still state of the art so many years later. It was back at the time where physicists did most calculations by hand. You'd have thought something more efficient would've turned out by now, since there was barely a sensible definition of efficiency back then.
- LeegleechN 4y ago
  Feynman diagrams are a representation of an integral so moving beyond them is a question of reformulating the mathematical basis of physics. There is some work in that direction (look up 'Amplituhedron') but overall it's not that surprising that physicists are using computers to do the same math with much more precision than they could before rather than shaking up the fundamental mathematical tools of their field which have stood since the 1600s.
zokier 4y ago
> Without ongoing development, FORM will get less and less usable — only able to interact with older computer code, and not aligned with how today’s students learn to program. Experienced users will stick with it, but younger researchers will adopt alternative computer algebra programs like Mathematica that are more user-friendly but orders of magnitude slower. In practice, many of these physicists will decide that certain problems are off-limits — too difficult to handle. So particle physics will stall, with only a few people able to work on the hardest calculations. This seems very pessimistic view. FORM was written by mortal humans (with much cruder tools!) in the first place, there is no reason to believe it couldn't be done again if need be and even improved in the process; its not some irreplaceable artifact descended from heavens. On the other hand, I imagine it would take bit more than just maintenance to make the 30 year old C codebase welcoming for new physicists to work on.
- xeonmc 4y ago
  I wouldn't be so sure about the march of progress being monotonously towards improvement. Take Electron,
  - a1369209993 4y ago
    > Take Electron, I'd rather not, thanks. Unless you meant in the sense of "take it out back and shoot it", in which case we're bloody trying already, it's not that easy.
- robacarp 4y ago
  I'm not at all worried that particle physics will slow down (...heh) because of this. It sounds like the one guy has been doing an incredible job of service maintenance over the years, sure, but these folks write software all the time. I have a hunch that FORM is not made up of easy to grok software designs or it would probably be true that other folks would have contributed in a bigger way regardless of academic stimuli. If that is indeed the case, as is common in research-academia software product, then tie a bow on it and send it off to the Smithsonian. I would be really quite surprised if some enterprising 19 year old whiz-bang genius from Iceland didn't re-write it in Mathematica.
avmich 4y ago
> The problem scales as a factorial, a mathematical relationship that grows even faster than exponential growth. Hm. I thought Stirling's approximation shows they are approximately of same growth?
- bigbillheck 4y ago
  Exponential is to factorial as linear is to log-linear.
fnordpiglet 4y ago
I feel like particle physics has gotten a ton of funding and there just be a lot of software engineering involved at the experimental level. I’m surprised if this software is that fundamental one or all of the funded experimental programs don’t break off some modest cash to pay people to improve and maintain it.
b34r 4y ago
Obsolescence means something else made it obsolete, meaning another program passed it technologically or provided superior user experience so usage naturally dropped off. It’s at risk of neglect/disrepair/falling behind. If it’s important enough and something breaks hard you better believe someone will pick it up, but not until it’s a forced issue.
bigbillheck 4y ago
Poking around the github there's a FORM cookbook: https://github.com/vermaseren/form/wiki/FORM-Cookbook https://github.com/vermaseren/form/wiki/FORM-Cookbook with some examples. It looks like it's basically a fancy term-rewriting system.
- ngcc_hk 4y ago
  Key is well used, well tested and the limit by disk not memory arrangement. I am not sure about the last bit. In 1970s they do have this issue and hence you have compiler that do multiple pass to handle limited memory. And there is no operating system to help, or there is no operating system (virtual memory, swapping, virtual machine come around that time …) wonder what is this application memory management scheme mentioned in the article is ? Overlay as in those days? https://en.wikipedia.org/wiki/Overlay_(programming) https://en.wikipedia.org/wiki/Overlay_(programming)

89 points by theafh 4y ago

Related discussion on programming and academia: https://news.ycombinator.com/item?id=33671264 https://news.ycombinator.com/item?id=33671264

prof-dr-ir 4y ago

https://github.com/vermaseren/form https://github.com/vermaseren/form

simne 4y ago

This is looking strange, because Particle Physics is one of very few areas, where hardware is not magnitudes cheaper than software. I mean not only computer hardware, which is also non cheap, but budgets for experimental hardware, which not all countries can handle. So, if program is really important, they could rearrange budgets and give some finances for software.

avidphantasm 4y ago
Agreed. I would be shocked if an entity like CERN would not be able to support something like FORM out of grant overhead.

moomin 4y ago

It's yet another story of the dumb misalign incentives in academia. Hand the guy a Nobel Prize. See if that moves the needle a bit.

Qem 4y ago

The root cause is academia still didn't adjust the rewards to take into account the computer age. People only get career points by publishing papers, not source code. Software is a fundamental aspect of most current research. We need to recognize researchers that dedicate to write/maintain the required software, awarding the fair amount of career points. Make good software documentation be the equivalent to papers for them, career-wise. It surely would improve the state of academic software. Today students still commonly have to struggle with software stuck in the 70s, usability-wise.

rowanG077 4y ago
It would also greatly increase the diffusion into industry of techniques. I often read papers that describe an interesting approach to something I am interested in doing. But often the paper requires me to read many references and spend dozens of hours or more to reach an implementation myself. I simply can't invest that time on something that might work. But if the code were available... then you could quickly test out if something is worth pursuing.
- Terretta 4y ago
  Arguably, by the time they implemented, they tested, at that point you're just consuming. So another way of handling this to contribute advancement at scale is "exercise left to the reader", and then we readers should publish too. R&D in big tech should contribute as academia does. We have our "Chief Scientists" and "Principal Engineers" titles that recognize, essentially, a tenure track, that should be on the merits of publishing code implementing new techniques.
Konohamaru 4y ago
This is a good idea. Imagine when future Nobel prizes are given to physicists who write and maintain software just as much to those who publish papers.
- Qem 4y ago
  This problem doesn't need any nobels to be solved. Just calibrate the rewards so programmer-scientists don't risk career stagnation if they take the time to develop and maintain tools of critical importance for science.
  - Konohamaru 4y ago
    But imagine if physicists were as recognized for software development as much as for experimental work or theorizing. That's a much more important reward.
4y ago

anigbrowl 4y ago

Universities tend to track scientists’ publication records Yet another example of administrators imposing a one-size-fits-all rule and elevating convenience over quality. Every academic I know hates how academia works, but they balk every time I suggest networking with people outside their disciplinary silos to dethrone the administrators. It's like the smartest people in the world are stymied by the idea of a collective action problem that requires working across disciplinary boundaries. For a long time I thought I was missing something fundamental about the complexity of the issue, but after 10 years of asking it seems to literally be that individuals are afraid of getting punished and they don't have any ideas beyond supporting the occasional strike action (that will leave things more or less as they are). I don't get it.

lowbloodsugar 4y ago
I mean you basically just described Autism. “Wow, you guys are amazing at your Special Interest and talking with other people with the same Special Interest. Why aren’t you interested at talking with these Neurotypical bean counters about finance stuff?” Not that I’m saying every physics major is Autistic. I myself know several in my graduating class who are not.
- anigbrowl 4y ago
  I resemble that remark myself. I would only clarify that I think the time for talking with bean counters is long past.
fnordpiglet 4y ago
Accountants know the price of everything and the value of nothing.

machina_ex_deus 4y ago

It's pretty remarkable that despite Feynman Diagrams being invented in the 1950s, before computers and computer science, they are still state of the art so many years later. It was back at the time where physicists did most calculations by hand. You'd have thought something more efficient would've turned out by now, since there was barely a sensible definition of efficiency back then.

LeegleechN 4y ago
Feynman diagrams are a representation of an integral so moving beyond them is a question of reformulating the mathematical basis of physics. There is some work in that direction (look up 'Amplituhedron') but overall it's not that surprising that physicists are using computers to do the same math with much more precision than they could before rather than shaking up the fundamental mathematical tools of their field which have stood since the 1600s.

zokier 4y ago

> Without ongoing development, FORM will get less and less usable — only able to interact with older computer code, and not aligned with how today’s students learn to program. Experienced users will stick with it, but younger researchers will adopt alternative computer algebra programs like Mathematica that are more user-friendly but orders of magnitude slower. In practice, many of these physicists will decide that certain problems are off-limits — too difficult to handle. So particle physics will stall, with only a few people able to work on the hardest calculations. This seems very pessimistic view. FORM was written by mortal humans (with much cruder tools!) in the first place, there is no reason to believe it couldn't be done again if need be and even improved in the process; its not some irreplaceable artifact descended from heavens. On the other hand, I imagine it would take bit more than just maintenance to make the 30 year old C codebase welcoming for new physicists to work on.

xeonmc 4y ago
I wouldn't be so sure about the march of progress being monotonously towards improvement. Take Electron,
- a1369209993 4y ago
  > Take Electron, I'd rather not, thanks. Unless you meant in the sense of "take it out back and shoot it", in which case we're bloody trying already, it's not that easy.
robacarp 4y ago
I'm not at all worried that particle physics will slow down (...heh) because of this. It sounds like the one guy has been doing an incredible job of service maintenance over the years, sure, but these folks write software all the time. I have a hunch that FORM is not made up of easy to grok software designs or it would probably be true that other folks would have contributed in a bigger way regardless of academic stimuli. If that is indeed the case, as is common in research-academia software product, then tie a bow on it and send it off to the Smithsonian. I would be really quite surprised if some enterprising 19 year old whiz-bang genius from Iceland didn't re-write it in Mathematica.

avmich 4y ago

> The problem scales as a factorial, a mathematical relationship that grows even faster than exponential growth. Hm. I thought Stirling's approximation shows they are approximately of same growth?

bigbillheck 4y ago
Exponential is to factorial as linear is to log-linear.

fnordpiglet 4y ago

I feel like particle physics has gotten a ton of funding and there just be a lot of software engineering involved at the experimental level. I’m surprised if this software is that fundamental one or all of the funded experimental programs don’t break off some modest cash to pay people to improve and maintain it.

b34r 4y ago

Obsolescence means something else made it obsolete, meaning another program passed it technologically or provided superior user experience so usage naturally dropped off. It’s at risk of neglect/disrepair/falling behind. If it’s important enough and something breaks hard you better believe someone will pick it up, but not until it’s a forced issue.

bigbillheck 4y ago

Poking around the github there's a FORM cookbook: https://github.com/vermaseren/form/wiki/FORM-Cookbook https://github.com/vermaseren/form/wiki/FORM-Cookbook with some examples. It looks like it's basically a fancy term-rewriting system.

ngcc_hk 4y ago
Key is well used, well tested and the limit by disk not memory arrangement. I am not sure about the last bit. In 1970s they do have this issue and hence you have compiler that do multiple pass to handle limited memory. And there is no operating system to help, or there is no operating system (virtual memory, swapping, virtual machine come around that time …) wonder what is this application memory management scheme mentioned in the article is ? Overlay as in those days? https://en.wikipedia.org/wiki/Overlay_(programming) https://en.wikipedia.org/wiki/Overlay_(programming)