4 ms·
This is an interesting article. I have often wondered if there would be benefits to porting scientific simulation codes to more modern languages. In the end I
by atrettel 6y ago
This is an interesting article. I have often wondered if there would be benefits to porting scientific simulation codes to more modern languages. In the end I've always reluctantly stayed in C or Fortran for scientific simulations. The article touched on the some of the issues (and the ever present issue of "If it isn't broke, don't fix it"). But I do think that the article ignored what I think is the biggest issue.
To me the biggest issue is that some simulations just need to run on 1000s or hundreds of 1000s of cores, and the primary technology that lets this happen is MPI (Message Passing Interface). I've been writing new Fortran code only because I want the simulation to be able to scale to this kind of size. The issue is that MPI only has official bindings for C and Fortran, so the only real choice I have is to keep programing in C or Fortran if I want to use MPI.
Any thoughts on this? Personally I would love to move towards more modern languages but without the proper libraries it is still a tough decision.
- zozbot234 6y agoRust can bind to C libraries, as can many other languages.
- atrettel 6y agoI have looked into this a bit, but the issue appears to be that MPI is not a library per se but a standard interface to more underlying function calls. Different implementations of MPI are in effect different libraries using the same API, and because of that it is difficult to write general bindings rather than just bindings to a particular version of an MPI library that is implemented on a particular supercomputer. In short, it's too site specific and fragile, unfortunately. If you change the system, the program might no longer compile. I appreciate the comment and feedback, though. This Stack Overflow page comments more on this issue: https://stackoverflow.com/questions/22949462/rust-on-grid-computing/24913447 https://stackoverflow.com/questions/22949462/rust-on-grid-co...
- TheRealKing 6y agoMPI officially supports only C and Fortran. Anything else for any other language can be only considered as adventures of some open-source developers, that may not last forever. Take a look at the fate of the C++ bindings for MPI.
- pjmlp 6y agoIncluding Fortran.
- jedimastert 6y agoI don't know that I have any thoughts, but I do have
- leephillips 6y agoConsider Julia. It has parallel processing built in, and uses MPI under the hood.
- dnautics 6y agoA bit more specifically: IIRC it defaults to trying to use MPI when you launch distributed jobs. It can also fall back to using SSH, for "extremely limited cases" and quite frankly it's not inappropriate to use SSH if you are certain that communication is not going to be a bottleneck.
- mfsch 6y agoFor those sorts of scientific simulations, you probably want to use the MPI.jl package rather than the built-in parallelism – at least that’s the conclusion I came to when I started the fluids code I’m currently working on. MPI.jl is a thin wrapper around the C interface so you end up writing your programs pretty much the same way as in C/Fortran. This type of MPI support is still a bit rough around the edges but it works, and our code runs as fast as the Fortran code it is based on (we usually scale to ~1000 cores). You get the advantage of working with a very nice, modern language, but it is pretty easy to accidentally do things that are very expensive and killing your performance, such as allocating memory inside a hot loop.
- jacobr1 6y agoWe use Akka in java for related distributed computation use cases.