4 ms·

It'll be interesting to see if Fortran makes a comeback. In some ways, modern Fortran is a kind of nice language. I'm mildly surprised for using Pytorch in HPC

by fault1 5y ago

It'll be interesting to see if Fortran makes a comeback. In some ways, modern Fortran is a kind of nice language. I'm mildly surprised for using Pytorch in HPC workloads since I didn't think its forward mode autodiff was great (unlike the reverse mode autodiff common in the NN world).

tubby12345 5y ago
Not sure what this means - pytorch's autodiff is reverse mode accumulation. You can read pazschke's paper on it or just go look at the implementation.
- fault1 5y ago
  I'm talking about forward - not reverse mode accumulation, since forward is VERY important for hpc. you ideally want 'forward over reverse' and other types of hybrid ad as well. it does look like a lot of progress has been made lately, but it isn't in a release yet: https://github.com/pytorch/pytorch/issues/10223 https://github.com/pytorch/pytorch/issues/10223
jamesfinlayson 5y ago
I doubt it will make a come-back, just because it's seen as ancient and out-dated. When was its hey-day anyway? Late 1980s or early 1990s maybe?
- findalex 5y ago
  Lot's of FORTRAN still in the LCF production queues.
- TheRealKing 5y ago
  I doubt your baseless doubt. What do you mean by outdated? do you mean those decades-old features like "modules" that other "modern" languages (like C++) are now adding in 2021? Or do you mean vectorized array syntax that is almost non-existent in other "modern" programming languages?, or do you mean a scalable human-readable built-in native passive parallel distributed communication paradigm that no other "modern" languages yet have? I assume it was a typo in your writing, you meant Fortran is a post-modern language.
  - jamesfinlayson 5y ago
    I mean it's not technology that most people will use in their day jobs, so in many companies it's unlikely to be suggested as a language to build something new in. I know one company in my area that has a Fortran codebase which continues to exist because it's critical software, but they continually struggle to find anyone who knows Fortran, let alone enough Fortran to work on such an important system.
- pjmlp 5y ago
  In HPC it never went away, it even got modules before C++.
pjmlp 5y ago
The latest standard is from 2018. https://fortranwiki.org/fortran/show/Fortran+2018 https://fortranwiki.org/fortran/show/Fortran+2018

by fault1 5y ago

tubby12345 5y ago

Not sure what this means - pytorch's autodiff is reverse mode accumulation. You can read pazschke's paper on it or just go look at the implementation.

fault1 5y ago
I'm talking about forward - not reverse mode accumulation, since forward is VERY important for hpc. you ideally want 'forward over reverse' and other types of hybrid ad as well. it does look like a lot of progress has been made lately, but it isn't in a release yet: https://github.com/pytorch/pytorch/issues/10223 https://github.com/pytorch/pytorch/issues/10223

jamesfinlayson 5y ago

I doubt it will make a come-back, just because it's seen as ancient and out-dated. When was its hey-day anyway? Late 1980s or early 1990s maybe?

findalex 5y ago
Lot's of FORTRAN still in the LCF production queues.
TheRealKing 5y ago
I doubt your baseless doubt. What do you mean by outdated? do you mean those decades-old features like "modules" that other "modern" languages (like C++) are now adding in 2021? Or do you mean vectorized array syntax that is almost non-existent in other "modern" programming languages?, or do you mean a scalable human-readable built-in native passive parallel distributed communication paradigm that no other "modern" languages yet have? I assume it was a typo in your writing, you meant Fortran is a post-modern language.
- jamesfinlayson 5y ago
  I mean it's not technology that most people will use in their day jobs, so in many companies it's unlikely to be suggested as a language to build something new in. I know one company in my area that has a Fortran codebase which continues to exist because it's critical software, but they continually struggle to find anyone who knows Fortran, let alone enough Fortran to work on such an important system.
pjmlp 5y ago
In HPC it never went away, it even got modules before C++.

pjmlp 5y ago

The latest standard is from 2018. https://fortranwiki.org/fortran/show/Fortran+2018 https://fortranwiki.org/fortran/show/Fortran+2018