5 ms·

Well, for one, this means that Apple Silicon has no Fortran compiler, which breaks a decent chunk of the scientific computing ecosystem.

dnautics 6y ago
is there not an flang project? Is it any good?
- KenoFischer 6y ago
  There's several! We tried all of them without much luck :(. The most complete seems to be the old flang, which was basically an open sourced version of the PGI compiler hacked up to generate LLVM IR by concating strings to generate textual LLVM IR. However, that project is basically abandoned and also depends on LLVM 7, while you probably want at least LLVM 10 to generate good code for Apple Silicon, so somebody would have to do that porting work, which doesn't seem like a worthwhile investment.
  - neurostimulant 6y ago
    Hopefully apple will do something about it as they did advertise their new system would be great for machine learning and all, which seem to be moot without many popular numeric libraries that directly or indirectly use fortran. Would be great if clang/llvm suddenly gain a great open source fortran compiler just because apple moves to arm.
analognoise 6y ago
Does anyone in the scientific computing ecosystem use Macs? I can't think of...any program in that sphere that aren't Windows or Linux?
- KenoFischer 6y ago
  Lots of students have Mac notebooks, but want to run julia/scientific python/etc.
- oefrha 6y ago
  Yes, that would be me, and half if not most of my colleagues.
- ephimetheus 6y ago
  Very much yes
- kxyvr 6y ago
  Sadly, yes. I've clients who still prefer to work on their Mac laptop and desktops. Large runs still go to Linux clusters, but small, debugging runs occur locally. Per the parent, the lack of Fortran compiler causes an issue because some BLAS/LAPACK libraries align their symbols with whatever a Fortran compiler would do. As such, unless there's been a recent change, things like CMake need the compiler to determine the name for linear algebra routines. I phrased the above as "sadly, yes" because supporting macos has been a pain and costs a fair amount of labor. In my case, Homebrew is required and there a bunch of specialized quirks in the build and packaging system that I'll contend are more difficult to deal with on macos than on Linux and Windows. Lastly, the requirement that macos be virtualized only on mac hardware complicates automating the build and test system since those processes must be relegated to a macos machine and I'd really rather not do primary development on a mac. Frankly, I'd pay Apple a not small amount of money to virtualize macos legally on non-mac hardware.
  - fluffything 6y ago
    BLAS/LAPACK isn't that much of an issue, since Apple probably will need to ship their own implementation of this for Apple Silicon to get reasonable performance, and they can implement this in C, assembly, or whatever they want. If they don't do that, then you can just use any "generic" C BLAS/LAPACK implementation, performance will be poor, but that's pretty much the only option if your hardware vendor doesn't support BLAS. I guess people can write their own BLAS as well, but if you want to re-implement BLAS in 2020, then Fortran is probably not the best language to do that.
    - KenoFischer 6y ago
      Apple's copy of LAPACK has not been keeping up with upstream and is missing some of the newer methods and bug fixes. Also while those are certainly the big two dependencies they are by far not the only ones.
- baggy_trough 6y ago
  Sure, lots.
- gumby 6y ago
  I see the opposite from you: Mac laptops, crunching under Linux, very rarely see windows in the wild. I know windows sales are still robust so this is what I see but am not sure how far it can be extrapolated.
- FridgeSeal 6y ago
  A huge chunk of scientific and numerical libraries rely on Fortran, for which 64-bit support is seriously lacking on Windows. Doing this work is significantly easier out-of-the-box on Mac/Linux device. Data Science is my main job and the first thing I did at my last job was ask if I could wipe windows and put Linux on the desktop they gave me.
  - pjmlp 6y ago
    Intel and PGI Fortran work just fine on Windows, with VS integration as pointed by others already.
  - ryl00 6y ago
    > A huge chunk of scientific and numerical libraries rely on Fortran, for which 64-bit support is seriously lacking on Windows. How so? mingw-64 exists, and so does Intel Fortran (with good integration with the VS IDE as well).
- washicalendar 6y ago
  Yes. Most people in my lab have a MacBook (Pro) where we do most of our coding, and we ssh into a Linux cluster to run most of the compute. We have desktops but these are all Windows/Linux dual boots. We all practically need to use PowerPoint as well, so Mac is nice to have here since it can’t go on Linux laptops, and nobody wants to constantly reboot all the time.
- eigenspace 6y ago
  Walking down the hallway in my physics department (pre-covid lockdown), I see almost all Macs. Some of them run windows, but most are apple hardware and software. Windows is quite rare when it comes to professors. It's more common among the grad students though, about tied with MacOS. It was like this in my previous department as well.
ArgyleSound 6y ago
Would it would work under Rosetta 2 (Apple's x64 emulation/translation layer) until the aarch64/darwin ABI can be ported to GCC? Or is there a reason to assume it might not?
- KenoFischer 6y ago
  Yes, it works mostly fine under Rosetta. However, it is also precisely the area where you don't really want a translation layer, because you're spending a lot of time doing heavy compute.
  - ArgyleSound 6y ago
    That makes sense. I'm hoping it's going to be an early adopter problem that'll have been solved by the time the transition is deemed complete.
  - sitkack 6y ago
    Many compute heavy tasks are using Metal and those should continue to function at the full speed of the hardware.
    - pritambaral 6y ago
      I doubt GCC is, or even could be, using Metal.
      - sitkack 6y ago
        I am referring to modern HPC like workloads, they will be using the GPU. Not saying GCC is using Metal in any way whatsoever.
        pritambaral 6y ago
        HPC on macOS? Using Metal instead of CUDA/OpenCL? Sounds extremely rare.
cat199 6y ago
Not sure if it's changed, but last i checked a few years back, most of the BLAS/LAPACK etc. stuff was f77, which should 'do' with f2c (albeit, perhaps not as fast) and is at least partly why f2c was hosted at netlib..
- sitkack 6y ago
  I had no idea, that explains so much! I was wondering how these open source numerical systems were building fortran code "so well", as in not a huge operational burden. My understanding is that the OSS fortran compilers are used for dev/debug, but when it comes to production, commercial Fortran compilers for your platform vendor offer much better performance.
bluejekyll 6y ago
Won’t this work? https://lfortran.org/ https://lfortran.org/ I just did a quick google search for it. I know nothing more than that it is FORTRAN compiler for LLVM. (It claims to be an alpha release)
- KenoFischer 6y ago
  Perhaps! It's one of the two candidates that might actually work in the near term. It didn't quite seem to work out of the box and given the newness of the project, we are a bit cautious, but there's some folks looking into it, to see if they can work through the issues.
fluffything 6y ago
This is incorrect, the LLVM project has an official Fortran compiler in tree called flang: https://github.com/llvm/llvm-project/tree/master/flang https://github.com/llvm/llvm-project/tree/master/flang It is quite hard to write portable Fortran code, but code that at least compiles with PGI should compile with that as well.
- sitkack 6y ago
  I wonder why this is allowed to persist? Have there been any studies across benchmark suites relative to compiler and platform? What was the spread? Was this general purpose, it works well enough speed or was this I need to get 1.75% better on task x or my super computer budget is blown? I find the whole Fortran ecosystem and lineage fascinating for some reason. Like an alternative universe where integers are only used for loop counters.
- KenoFischer 6y ago
  If you actually read the code, it turns out that said compiler is not yet capable of generating LLVM IR let alone binaries.
  - fluffything 6y ago
    Wait what? We were using the old flang project just fine. I thought they just moved it in tree and keept developing it.
    - KenoFischer 6y ago
      There was old flang which was basically just the PGI compiler stripped of its backend and with a piece bolted on that converted the compiler's internal IR to LLVM by basically dumping out strings and then parsing them back in. Many people in the LLVM community thought that this was not a maintainable long term strategy for a Fortran frontend, and after much debate, the flang team decided to start over on Greenfield compiler called f18, which was to be architected more like clang. Then Chris released MLIR and said team decided to target that rather than LLVM directly. The only thing that was kept from old flang were some of the runtime libraries. f18 was recently merged into the llvm-project repository whereupon it was renamed flang. Old flang is dead, but the terminology is a bit confusing now. Additionally there are two external LLVM Fortran compiler efforts: lfortran out of the US national labs (which also paid PGI for flang) and fc, which I don't believe is currently open source, but is being developed by as compiler consulting shop on India called CompilerTree (not sure who's footing the bill for that one). Anyway, long story short, none of them seem to work. Old flang generates binaries, but is abandonware at this point, so not the correct foundation to build on. New flang doesn't even generate LLVM IR yet. FC doesn't seem public yet as I said. Lfortran may work, but the authors say it's very alpha and it certainly didn't work out of the box. I know some folks are looking into it to see what would be required.
TheRealKing 6y ago
Apple sucks for scientific computing. It does not let you even load your own generated shared files, because Apple determines it is too dangerous for the health my own purchased computer. I have never had that problem on Windows or Linux. That said, you can get gfortran by as simple as typing brew install gcc-9, or some similar command (requires brew installation).

dnautics 6y ago

is there not an flang project? Is it any good?

KenoFischer 6y ago
There's several! We tried all of them without much luck :(. The most complete seems to be the old flang, which was basically an open sourced version of the PGI compiler hacked up to generate LLVM IR by concating strings to generate textual LLVM IR. However, that project is basically abandoned and also depends on LLVM 7, while you probably want at least LLVM 10 to generate good code for Apple Silicon, so somebody would have to do that porting work, which doesn't seem like a worthwhile investment.
- neurostimulant 6y ago
  Hopefully apple will do something about it as they did advertise their new system would be great for machine learning and all, which seem to be moot without many popular numeric libraries that directly or indirectly use fortran. Would be great if clang/llvm suddenly gain a great open source fortran compiler just because apple moves to arm.

analognoise 6y ago

Does anyone in the scientific computing ecosystem use Macs? I can't think of...any program in that sphere that aren't Windows or Linux?

KenoFischer 6y ago
Lots of students have Mac notebooks, but want to run julia/scientific python/etc.
oefrha 6y ago
Yes, that would be me, and half if not most of my colleagues.
ephimetheus 6y ago
Very much yes
kxyvr 6y ago
Sadly, yes. I've clients who still prefer to work on their Mac laptop and desktops. Large runs still go to Linux clusters, but small, debugging runs occur locally. Per the parent, the lack of Fortran compiler causes an issue because some BLAS/LAPACK libraries align their symbols with whatever a Fortran compiler would do. As such, unless there's been a recent change, things like CMake need the compiler to determine the name for linear algebra routines. I phrased the above as "sadly, yes" because supporting macos has been a pain and costs a fair amount of labor. In my case, Homebrew is required and there a bunch of specialized quirks in the build and packaging system that I'll contend are more difficult to deal with on macos than on Linux and Windows. Lastly, the requirement that macos be virtualized only on mac hardware complicates automating the build and test system since those processes must be relegated to a macos machine and I'd really rather not do primary development on a mac. Frankly, I'd pay Apple a not small amount of money to virtualize macos legally on non-mac hardware.
- fluffything 6y ago
  BLAS/LAPACK isn't that much of an issue, since Apple probably will need to ship their own implementation of this for Apple Silicon to get reasonable performance, and they can implement this in C, assembly, or whatever they want. If they don't do that, then you can just use any "generic" C BLAS/LAPACK implementation, performance will be poor, but that's pretty much the only option if your hardware vendor doesn't support BLAS. I guess people can write their own BLAS as well, but if you want to re-implement BLAS in 2020, then Fortran is probably not the best language to do that.
  - KenoFischer 6y ago
    Apple's copy of LAPACK has not been keeping up with upstream and is missing some of the newer methods and bug fixes. Also while those are certainly the big two dependencies they are by far not the only ones.
baggy_trough 6y ago
Sure, lots.
gumby 6y ago
I see the opposite from you: Mac laptops, crunching under Linux, very rarely see windows in the wild. I know windows sales are still robust so this is what I see but am not sure how far it can be extrapolated.
FridgeSeal 6y ago
A huge chunk of scientific and numerical libraries rely on Fortran, for which 64-bit support is seriously lacking on Windows. Doing this work is significantly easier out-of-the-box on Mac/Linux device. Data Science is my main job and the first thing I did at my last job was ask if I could wipe windows and put Linux on the desktop they gave me.
- pjmlp 6y ago
  Intel and PGI Fortran work just fine on Windows, with VS integration as pointed by others already.
- ryl00 6y ago
  > A huge chunk of scientific and numerical libraries rely on Fortran, for which 64-bit support is seriously lacking on Windows. How so? mingw-64 exists, and so does Intel Fortran (with good integration with the VS IDE as well).
washicalendar 6y ago
Yes. Most people in my lab have a MacBook (Pro) where we do most of our coding, and we ssh into a Linux cluster to run most of the compute. We have desktops but these are all Windows/Linux dual boots. We all practically need to use PowerPoint as well, so Mac is nice to have here since it can’t go on Linux laptops, and nobody wants to constantly reboot all the time.
eigenspace 6y ago
Walking down the hallway in my physics department (pre-covid lockdown), I see almost all Macs. Some of them run windows, but most are apple hardware and software. Windows is quite rare when it comes to professors. It's more common among the grad students though, about tied with MacOS. It was like this in my previous department as well.

ArgyleSound 6y ago

Would it would work under Rosetta 2 (Apple's x64 emulation/translation layer) until the aarch64/darwin ABI can be ported to GCC? Or is there a reason to assume it might not?

KenoFischer 6y ago
Yes, it works mostly fine under Rosetta. However, it is also precisely the area where you don't really want a translation layer, because you're spending a lot of time doing heavy compute.
- ArgyleSound 6y ago
  That makes sense. I'm hoping it's going to be an early adopter problem that'll have been solved by the time the transition is deemed complete.
- sitkack 6y ago
  Many compute heavy tasks are using Metal and those should continue to function at the full speed of the hardware.
  - pritambaral 6y ago
    I doubt GCC is, or even could be, using Metal.
    - sitkack 6y ago
      I am referring to modern HPC like workloads, they will be using the GPU. Not saying GCC is using Metal in any way whatsoever.
      - pritambaral 6y ago
        HPC on macOS? Using Metal instead of CUDA/OpenCL? Sounds extremely rare.

cat199 6y ago

Not sure if it's changed, but last i checked a few years back, most of the BLAS/LAPACK etc. stuff was f77, which should 'do' with f2c (albeit, perhaps not as fast) and is at least partly why f2c was hosted at netlib..

sitkack 6y ago
I had no idea, that explains so much! I was wondering how these open source numerical systems were building fortran code "so well", as in not a huge operational burden. My understanding is that the OSS fortran compilers are used for dev/debug, but when it comes to production, commercial Fortran compilers for your platform vendor offer much better performance.

bluejekyll 6y ago

Won’t this work? https://lfortran.org/ https://lfortran.org/ I just did a quick google search for it. I know nothing more than that it is FORTRAN compiler for LLVM. (It claims to be an alpha release)

KenoFischer 6y ago
Perhaps! It's one of the two candidates that might actually work in the near term. It didn't quite seem to work out of the box and given the newness of the project, we are a bit cautious, but there's some folks looking into it, to see if they can work through the issues.

fluffything 6y ago

This is incorrect, the LLVM project has an official Fortran compiler in tree called flang: https://github.com/llvm/llvm-project/tree/master/flang https://github.com/llvm/llvm-project/tree/master/flang It is quite hard to write portable Fortran code, but code that at least compiles with PGI should compile with that as well.

sitkack 6y ago
I wonder why this is allowed to persist? Have there been any studies across benchmark suites relative to compiler and platform? What was the spread? Was this general purpose, it works well enough speed or was this I need to get 1.75% better on task x or my super computer budget is blown? I find the whole Fortran ecosystem and lineage fascinating for some reason. Like an alternative universe where integers are only used for loop counters.
KenoFischer 6y ago
If you actually read the code, it turns out that said compiler is not yet capable of generating LLVM IR let alone binaries.
- fluffything 6y ago
  Wait what? We were using the old flang project just fine. I thought they just moved it in tree and keept developing it.
  - KenoFischer 6y ago
    There was old flang which was basically just the PGI compiler stripped of its backend and with a piece bolted on that converted the compiler's internal IR to LLVM by basically dumping out strings and then parsing them back in. Many people in the LLVM community thought that this was not a maintainable long term strategy for a Fortran frontend, and after much debate, the flang team decided to start over on Greenfield compiler called f18, which was to be architected more like clang. Then Chris released MLIR and said team decided to target that rather than LLVM directly. The only thing that was kept from old flang were some of the runtime libraries. f18 was recently merged into the llvm-project repository whereupon it was renamed flang. Old flang is dead, but the terminology is a bit confusing now. Additionally there are two external LLVM Fortran compiler efforts: lfortran out of the US national labs (which also paid PGI for flang) and fc, which I don't believe is currently open source, but is being developed by as compiler consulting shop on India called CompilerTree (not sure who's footing the bill for that one). Anyway, long story short, none of them seem to work. Old flang generates binaries, but is abandonware at this point, so not the correct foundation to build on. New flang doesn't even generate LLVM IR yet. FC doesn't seem public yet as I said. Lfortran may work, but the authors say it's very alpha and it certainly didn't work out of the box. I know some folks are looking into it to see what would be required.

TheRealKing 6y ago

Apple sucks for scientific computing. It does not let you even load your own generated shared files, because Apple determines it is too dangerous for the health my own purchased computer. I have never had that problem on Windows or Linux. That said, you can get gfortran by as simple as typing brew install gcc-9, or some similar command (requires brew installation).