12 ms·

NASA: High Performance Fast Computing Challenge

125 points by happy-go-lucky 10y ago

mynegation 10y ago
Putting up two TL;DR filters right away: US citizens only, Fortran codebase :-/
- 52-6F-62 10y ago
  > US citizens only Big frown over here
- snovv_crash 10y ago
  How about US expats?
  - cbcoutinho 10y ago
    Are you still a US citizen?
- gravypod 10y ago
  There is sadly a huge amount of Fortran code bases in science. Not old. Not left over libraries. Not software written for a bygone age. No... New software. Huge, mission critical, core project, software. The best part: No documentation, no maintainer, and no migration plan. Scientists think in terms of how Units of Science per Units of Work. Translating code, learning new languages, testing, checking correctness, software validation, bug testing, or even just using external (non-science) libraries yield a very low science/work ratio. Just opening a text editor, writing a language you already know, and calling it "good enough" is high science/work. Their method of software validation usually boils down to "spit out your data, graph it, and see if it looks like what you expect". There's also this notion that "Fortran is fast" that MANY people hold for some reason. They don't know about what kinds of modern compiler optimizations they're missing out on and what new IPC frameworks are available. They know Fortran is fast, MPI is parallel, and that's what they use. Right now I'm looking at a codebase that is actively being developed and is written in Fortran. It's about 48,466 lines in total. The source is about 2.3M in size. No one seems to have formula listings for it. I want to rewrite it but pulling apart fortran is very difficult. It's very scary that if I were to write the software in C that it would be seen as modern! This software is tax-payer funded and included in real time systems and production software. (On that note if anyone is good at Fortran and can document the formulas in this software so I can rewrite them I'd be happy! Please email me)
  - DHermit 10y ago
    What makes you think that Fortran is not a modern language? And when writing scientific code which involves multidimensional matrices, Fortran is most of the time more readable than C, because you can do array slicing, use elemental functions over an array (like simply doing sin(a) where a is an array). Your whole post is just complaining about Fortran still beeing used, but you don't really explain why. The only point I can find is > The best part: No documentation, no maintainer, and no migration plan. which is in no way language specific. Have you looked at modern Fortran? You have object orientation, polymophism and even parallel features like coarrays (I have to admit that I never really used them, because the gcc support is somehow incomplete).
  - goalieca 10y ago
    I used to think academic code was a hack fest. Then I entered industry. I would not be so quick to shit on scientists. They may not be language gurus writing with the latest idioms and toolkits but there is a lot of carefully analyzed and "correct" software.
  - qb45 10y ago
    > There's also this notion that "Fortran is fast" that MANY people hold for some reason. They don't know about what kinds of modern compiler optimizations they're missing out on and what new IPC frameworks are available. Fortran is fast and has no reason to be worse than C. In fact, it may be easier to write compiler-friendly code in F thanks to "restrict by default". And are there really HPC-grade C compilers with whatever "modern optimizations" that don't come with a Fortran frontend? Don't think so. BTW, I have seen an attempt to rewrite a few dozen KLOC Fortran project to C and it wasn't pretty. The code was still a bunch of for loops crunching through a bunch of arrays, just in a different language, and then it had some new bugs that were introduced in translation and also it produced slightly different results because of different language rules, compiler and optimizations translating the same math formulas to different machine code. And the formulas were written to produce expected output when compiled with, you guessed it, Fortran. Don't get me wrong, the old-school Fortran with 3-letter variable names, 72 character columns, GOTOs and stuff sucks donkey balls. But you don't have to write like that - while I'm no real Fortran expert (sorry), I've heard that modern standards of Fortran are vastly better and look much closer to C. I would certainly research this before rewriting everything in C.
- walshemj 10y ago
  Why for serious CFD / HPC Fortran is still the daddy
m-can 10y ago
Does the phrase “nonlinear partial differential equations used for unsteady computations” excite you?
- tanderson92 10y ago
  Yes. But then that's what my Ph.D. thesis area currently is.
- arcanus 10y ago
  On HN it might be a sizable percentage.
StreamBright 10y ago
I got really excited until I reached the US citizens only clause. I wish it was not export regulated, the problem domain sounds pretty interesting.
- georgiev 10y ago
  Why does it always have to be US-citizen only. FeelsBadMan
  - rtkwe 10y ago
    Because the difference between a rocket and an ICBM is basically non existent and being able to simulate the environment inside a rocket engine would help a lot with designing either. To do pretty much any job in aerospace you'll have to qualify under ITAR which means you'll have to be a U.S. citizen, lawful permanent resident of the U.S., protected individual as defined by 8 U.S.C. 1324b(a)(3), or eligible to obtain the required authorizations from the U.S. Department of State.
    - devrandomguy 10y ago
      So, why don't we have these restrictions in Canada, Japan or Europe? Many countries have advanced aerospace programs. The only reason that I can see for those export regulations to exist, is because eliminating them would require effort and PR risk on the part of politicians.
      - rtkwe 10y ago
        ITAR is a little old so it's a bit crufty from a different time. But also I'd be shocked if ESA, JAXA, and CSA were completely open.
    - StreamBright 10y ago
      True, however there are other countries that are considered allies of US at the extent that new technology is ok to be worked on (more specifically HPC). https://fas.org/sgp/crs/natsec/RL31175.pdf https://fas.org/sgp/crs/natsec/RL31175.pdf
dzdt 10y ago
The prizes seem pretty low to me: $15k top prize for several months of work requiring a skillset that can easily land a 150k-plus-per-year salary. And no guarantees you get anything for your efforts. Edit: on the other hand graduate students and postdocs are notoriously underpaid and often have a lot of freedom in how they spend their summer time. I guess that is the target competitor pool?
- goalieca 10y ago
  Maybe there's a paper waiting for the grad student who cracks the problem?
- walshemj 10y ago
  And it doesn't say which Fortran they are using bog standard GFortran or Intel Fortran with all the parallel extensions. With this sort of code you also tune it to the machine architecture back when I did F77 for BT / Dialcom the code was tweaked to suit the PR!ME Architecture to get extra speed
  - jheriko 10y ago
    when they say 'modern' fortran i take it to imply a a compiler and standard from sometime in the last 20 years. i've heard people refer to c90 as modern before.. (in the last two years) and the nasa/astronomy guys are famously clingy to 'known good' tech from the olden days. only the very most recent astronomical dataset i acquired was in .csv. everything is made for fortran from what i've seen over the last 10 years...
- azhenley 10y ago
  Grad students in CS can easily make over $30k at a summer internship (I've done several).
- tanderson92 10y ago
  The graduate students who can solve and win this problem are not underpaid.
- photon-torpedo 10y ago
deleted 10y ago
[deleted]
jacknews 10y ago
Hey geeks! NASA!! Aircraft design!! Spaaaace!!!!! Just optimize our old code, and we'll pay peanuts! If you win, that is. We were going to hire someone to shephard the old code, but why bother when you geeks will do it for nearly free because it's so cool!!! OK I have my cynical hat on, and I can't really blame them for trying a cost-effective approach, but is this the future of work?
- tanderson92 10y ago
  Not if the people who could solve this problem refuse to participate.
- msl 10y ago
  Well, this is NASA. They are not exactly rolling in cash. Maybe you could look at it as a sort of a voluntarily paid tax, earmarked for space exploration? I, for one, would be happy to donate some of my time for the cause, if they were willing to let me. > is this the future of work? Not just the future: many people have contributed to open source projects that have ended up being used in commercial projects, and never received any payment for their work.
  - jacknews 10y ago
    Good point, though I see open source as something people create for pleasure, rather than 'solving someone else's problem'. There is much blur between those two things obviously.
Twirrim 10y ago
This will add complexity to the challenge somewhat: The cluster uses a mix of E5-2680v4 (Broadwell), E5-2680v3 (Haswell), E5-2680v2 (Ivy Bridge), and E5-2670 (Sandy Bridge) processors. Each generation has brought different optimisations, tweaked instruction sets, and performance characteristics. You'll have to optimise for the most common scenario.
- rootbear 10y ago
  This is a common problem with modern supercomputers, which are all clusters of some sort. We have a supercomputer at NASA Goddard, much smaller than Pleiades, that gets new "Scalable Units" every few years. The first parts of it are now long gone and what's there is a mix of several recent generations of Intel CPU and some Intel Phis. I'm told that some codes run better on the older CPUs due to caching and other architectural issues.
analognoise 10y ago
So uh...anyone want to team up on this one?
soreal 10y ago
They're giving you a chance to win $15k and all it takes is delivering a 1000x speedup improvement to software that runs on a 250k CPU cluster? I'll pass, but hopefully someone else can do this as a labor of love.
- photon-torpedo 10y ago
  > runs on a 250k CPU cluster Yes, except you are not allowed to run the software on their cluster, instead you are expected to run it on your own machine.
devrandomguy 10y ago
> As a participant, you’ll need to gain access to FUN3D software through an application process with the US Government. This is not at all ready for a competition. The least they could do, if they want professional volunteer help, is to open-source the platform that everyone would be deploying on. It was probably built using public money, after all. Sigh, no shame: > FUN3D is: Code developed by the US Government at US taxpayer expense, ...
- martijn_himself 10y ago
  As a Dutch national living in the UK I did get somewhat excited to at least have a look at the code. It is always disheartening to read these exclusion statements for a institution with a global appeal like NASA. At the very least its bad press for them in a global science community. I'm also somewhat amused by the claim they expect a 1000x performance improvement.
- flamedoge 10y ago
  So much talent out there yet limited to US citizens only
devrandomguy 10y ago
Is anyone here able to compare this FUN3D CFD solver with open source CFD software, such as PyFR, Fluidity or OpenFOAM? In particular, has anyone done any benchmark comparisons? I'm wondering if maybe NASA would be better off over the long term, if they ported their models to a more accessible platform. It sounds like they are having trouble hiring people who can work on their current stack.
- fcanesin 10y ago
  This opensource libraries that you mention (and most in general) are focused in generic constructs of methods for the solution of partial differential equations (PDEs). PyFR - flux reconstruction. OpenFOAM - Finite Volume. The list is actually extensive, there are libraries with focus in Finite Element, Finite Difference, Boundary Element, Smoothed Particle hydrodynamics and so on. The difference here is that this is a solver, not a library. So it is focused on a specific set of PDEs, by doing so you can apply optimizations that are specific to this PDEs and the space representation that it uses, this optimizations are not general to the underlying method of solution (which can actually be a combination of methods), so it will be extremely hard for this generic libraries to compete in performance with a focused solver.
jxy 10y ago
Am I reading it correctly? They want people to optimize their code without directly accessing their hardware. This is not how it should be done. Is it really from NASA? Are they lacking HPC programmers? Or any one knows HPC? Either their code is really bad such that optimizing on your home commodity computer is enough to get it run faster, or they are just making fun of it. Even if the code gets much faster with their aging clusters, it wouldn't necessary be any better with skylake or knights landing.
lightbendover 10y ago
Accuracy is only 20% of the puzzle so you could probably grab an easy 80% score by just returning some random numbers right off the bat. Well, now only 70% because you're not going to get the Originality portion anymore. Even without the tongue-in-cheek, this seems like a sloppily concocted competition, but good luck to those who play around with it.
cube00 10y ago
Nice, they've worked out how to take away their developer's down time. > To catch integration errors, the suite of codes are repeatedly checked-out around the clock from our central repository, compiled, and several hundred regression and unit tests are run. Email and cell phone SMS provide instant notification of any errors. https://fun3d.larc.nasa.gov/chapter-1.html https://fun3d.larc.nasa.gov/chapter-1.html
jheriko 10y ago
tbh the prizes are well below par. :/ also their us government tie... this is just a coded message for me that i am not welcome because i am too arabic. at least at first glance. probably its not but this is my experience of trying to do anything involving america in the last several years (i can't speak about the trump administration - but i doubt they are more arab friendly given the bad press).

125 points by happy-go-lucky 10y ago

mynegation 10y ago

Putting up two TL;DR filters right away: US citizens only, Fortran codebase :-/

52-6F-62 10y ago
> US citizens only Big frown over here
snovv_crash 10y ago
How about US expats?
- cbcoutinho 10y ago
  Are you still a US citizen?
gravypod 10y ago
There is sadly a huge amount of Fortran code bases in science. Not old. Not left over libraries. Not software written for a bygone age. No... New software. Huge, mission critical, core project, software. The best part: No documentation, no maintainer, and no migration plan. Scientists think in terms of how Units of Science per Units of Work. Translating code, learning new languages, testing, checking correctness, software validation, bug testing, or even just using external (non-science) libraries yield a very low science/work ratio. Just opening a text editor, writing a language you already know, and calling it "good enough" is high science/work. Their method of software validation usually boils down to "spit out your data, graph it, and see if it looks like what you expect". There's also this notion that "Fortran is fast" that MANY people hold for some reason. They don't know about what kinds of modern compiler optimizations they're missing out on and what new IPC frameworks are available. They know Fortran is fast, MPI is parallel, and that's what they use. Right now I'm looking at a codebase that is actively being developed and is written in Fortran. It's about 48,466 lines in total. The source is about 2.3M in size. No one seems to have formula listings for it. I want to rewrite it but pulling apart fortran is very difficult. It's very scary that if I were to write the software in C that it would be seen as modern! This software is tax-payer funded and included in real time systems and production software. (On that note if anyone is good at Fortran and can document the formulas in this software so I can rewrite them I'd be happy! Please email me)
- DHermit 10y ago
  What makes you think that Fortran is not a modern language? And when writing scientific code which involves multidimensional matrices, Fortran is most of the time more readable than C, because you can do array slicing, use elemental functions over an array (like simply doing sin(a) where a is an array). Your whole post is just complaining about Fortran still beeing used, but you don't really explain why. The only point I can find is > The best part: No documentation, no maintainer, and no migration plan. which is in no way language specific. Have you looked at modern Fortran? You have object orientation, polymophism and even parallel features like coarrays (I have to admit that I never really used them, because the gcc support is somehow incomplete).
- goalieca 10y ago
  I used to think academic code was a hack fest. Then I entered industry. I would not be so quick to shit on scientists. They may not be language gurus writing with the latest idioms and toolkits but there is a lot of carefully analyzed and "correct" software.
- qb45 10y ago
  > There's also this notion that "Fortran is fast" that MANY people hold for some reason. They don't know about what kinds of modern compiler optimizations they're missing out on and what new IPC frameworks are available. Fortran is fast and has no reason to be worse than C. In fact, it may be easier to write compiler-friendly code in F thanks to "restrict by default". And are there really HPC-grade C compilers with whatever "modern optimizations" that don't come with a Fortran frontend? Don't think so. BTW, I have seen an attempt to rewrite a few dozen KLOC Fortran project to C and it wasn't pretty. The code was still a bunch of for loops crunching through a bunch of arrays, just in a different language, and then it had some new bugs that were introduced in translation and also it produced slightly different results because of different language rules, compiler and optimizations translating the same math formulas to different machine code. And the formulas were written to produce expected output when compiled with, you guessed it, Fortran. Don't get me wrong, the old-school Fortran with 3-letter variable names, 72 character columns, GOTOs and stuff sucks donkey balls. But you don't have to write like that - while I'm no real Fortran expert (sorry), I've heard that modern standards of Fortran are vastly better and look much closer to C. I would certainly research this before rewriting everything in C.
walshemj 10y ago
Why for serious CFD / HPC Fortran is still the daddy

m-can 10y ago

Does the phrase “nonlinear partial differential equations used for unsteady computations” excite you?

tanderson92 10y ago
Yes. But then that's what my Ph.D. thesis area currently is.
arcanus 10y ago
On HN it might be a sizable percentage.

StreamBright 10y ago

I got really excited until I reached the US citizens only clause. I wish it was not export regulated, the problem domain sounds pretty interesting.

georgiev 10y ago
Why does it always have to be US-citizen only. FeelsBadMan
- rtkwe 10y ago
  Because the difference between a rocket and an ICBM is basically non existent and being able to simulate the environment inside a rocket engine would help a lot with designing either. To do pretty much any job in aerospace you'll have to qualify under ITAR which means you'll have to be a U.S. citizen, lawful permanent resident of the U.S., protected individual as defined by 8 U.S.C. 1324b(a)(3), or eligible to obtain the required authorizations from the U.S. Department of State.
  - devrandomguy 10y ago
    So, why don't we have these restrictions in Canada, Japan or Europe? Many countries have advanced aerospace programs. The only reason that I can see for those export regulations to exist, is because eliminating them would require effort and PR risk on the part of politicians.
    - rtkwe 10y ago
      ITAR is a little old so it's a bit crufty from a different time. But also I'd be shocked if ESA, JAXA, and CSA were completely open.
  - StreamBright 10y ago
    True, however there are other countries that are considered allies of US at the extent that new technology is ok to be worked on (more specifically HPC). https://fas.org/sgp/crs/natsec/RL31175.pdf https://fas.org/sgp/crs/natsec/RL31175.pdf

dzdt 10y ago

The prizes seem pretty low to me: $15k top prize for several months of work requiring a skillset that can easily land a 150k-plus-per-year salary. And no guarantees you get anything for your efforts. Edit: on the other hand graduate students and postdocs are notoriously underpaid and often have a lot of freedom in how they spend their summer time. I guess that is the target competitor pool?

goalieca 10y ago
Maybe there's a paper waiting for the grad student who cracks the problem?
walshemj 10y ago
And it doesn't say which Fortran they are using bog standard GFortran or Intel Fortran with all the parallel extensions. With this sort of code you also tune it to the machine architecture back when I did F77 for BT / Dialcom the code was tweaked to suit the PR!ME Architecture to get extra speed
- jheriko 10y ago
  when they say 'modern' fortran i take it to imply a a compiler and standard from sometime in the last 20 years. i've heard people refer to c90 as modern before.. (in the last two years) and the nasa/astronomy guys are famously clingy to 'known good' tech from the olden days. only the very most recent astronomical dataset i acquired was in .csv. everything is made for fortran from what i've seen over the last 10 years...
azhenley 10y ago
Grad students in CS can easily make over $30k at a summer internship (I've done several).
tanderson92 10y ago
The graduate students who can solve and win this problem are not underpaid.
photon-torpedo 10y ago

deleted 10y ago

[deleted]

jacknews 10y ago

Hey geeks! NASA!! Aircraft design!! Spaaaace!!!!! Just optimize our old code, and we'll pay peanuts! If you win, that is. We were going to hire someone to shephard the old code, but why bother when you geeks will do it for nearly free because it's so cool!!! OK I have my cynical hat on, and I can't really blame them for trying a cost-effective approach, but is this the future of work?

tanderson92 10y ago
Not if the people who could solve this problem refuse to participate.
msl 10y ago
Well, this is NASA. They are not exactly rolling in cash. Maybe you could look at it as a sort of a voluntarily paid tax, earmarked for space exploration? I, for one, would be happy to donate some of my time for the cause, if they were willing to let me. > is this the future of work? Not just the future: many people have contributed to open source projects that have ended up being used in commercial projects, and never received any payment for their work.
- jacknews 10y ago
  Good point, though I see open source as something people create for pleasure, rather than 'solving someone else's problem'. There is much blur between those two things obviously.

Twirrim 10y ago

This will add complexity to the challenge somewhat: The cluster uses a mix of E5-2680v4 (Broadwell), E5-2680v3 (Haswell), E5-2680v2 (Ivy Bridge), and E5-2670 (Sandy Bridge) processors. Each generation has brought different optimisations, tweaked instruction sets, and performance characteristics. You'll have to optimise for the most common scenario.

rootbear 10y ago
This is a common problem with modern supercomputers, which are all clusters of some sort. We have a supercomputer at NASA Goddard, much smaller than Pleiades, that gets new "Scalable Units" every few years. The first parts of it are now long gone and what's there is a mix of several recent generations of Intel CPU and some Intel Phis. I'm told that some codes run better on the older CPUs due to caching and other architectural issues.

analognoise 10y ago

So uh...anyone want to team up on this one?

soreal 10y ago

They're giving you a chance to win $15k and all it takes is delivering a 1000x speedup improvement to software that runs on a 250k CPU cluster? I'll pass, but hopefully someone else can do this as a labor of love.

photon-torpedo 10y ago
> runs on a 250k CPU cluster Yes, except you are not allowed to run the software on their cluster, instead you are expected to run it on your own machine.

devrandomguy 10y ago

> As a participant, you’ll need to gain access to FUN3D software through an application process with the US Government. This is not at all ready for a competition. The least they could do, if they want professional volunteer help, is to open-source the platform that everyone would be deploying on. It was probably built using public money, after all. Sigh, no shame: > FUN3D is: Code developed by the US Government at US taxpayer expense, ...

martijn_himself 10y ago
As a Dutch national living in the UK I did get somewhat excited to at least have a look at the code. It is always disheartening to read these exclusion statements for a institution with a global appeal like NASA. At the very least its bad press for them in a global science community. I'm also somewhat amused by the claim they expect a 1000x performance improvement.
flamedoge 10y ago
So much talent out there yet limited to US citizens only

devrandomguy 10y ago

Is anyone here able to compare this FUN3D CFD solver with open source CFD software, such as PyFR, Fluidity or OpenFOAM? In particular, has anyone done any benchmark comparisons? I'm wondering if maybe NASA would be better off over the long term, if they ported their models to a more accessible platform. It sounds like they are having trouble hiring people who can work on their current stack.

fcanesin 10y ago
This opensource libraries that you mention (and most in general) are focused in generic constructs of methods for the solution of partial differential equations (PDEs). PyFR - flux reconstruction. OpenFOAM - Finite Volume. The list is actually extensive, there are libraries with focus in Finite Element, Finite Difference, Boundary Element, Smoothed Particle hydrodynamics and so on. The difference here is that this is a solver, not a library. So it is focused on a specific set of PDEs, by doing so you can apply optimizations that are specific to this PDEs and the space representation that it uses, this optimizations are not general to the underlying method of solution (which can actually be a combination of methods), so it will be extremely hard for this generic libraries to compete in performance with a focused solver.

jxy 10y ago

Am I reading it correctly? They want people to optimize their code without directly accessing their hardware. This is not how it should be done. Is it really from NASA? Are they lacking HPC programmers? Or any one knows HPC? Either their code is really bad such that optimizing on your home commodity computer is enough to get it run faster, or they are just making fun of it. Even if the code gets much faster with their aging clusters, it wouldn't necessary be any better with skylake or knights landing.

lightbendover 10y ago

Accuracy is only 20% of the puzzle so you could probably grab an easy 80% score by just returning some random numbers right off the bat. Well, now only 70% because you're not going to get the Originality portion anymore. Even without the tongue-in-cheek, this seems like a sloppily concocted competition, but good luck to those who play around with it.

cube00 10y ago

Nice, they've worked out how to take away their developer's down time. > To catch integration errors, the suite of codes are repeatedly checked-out around the clock from our central repository, compiled, and several hundred regression and unit tests are run. Email and cell phone SMS provide instant notification of any errors. https://fun3d.larc.nasa.gov/chapter-1.html https://fun3d.larc.nasa.gov/chapter-1.html

jheriko 10y ago

tbh the prizes are well below par. :/ also their us government tie... this is just a coded message for me that i am not welcome because i am too arabic. at least at first glance. probably its not but this is my experience of trying to do anything involving america in the last several years (i can't speak about the trump administration - but i doubt they are more arab friendly given the bad press).