4 ms·

I have to be honest with you, as someone who writes in both C++ and Python, I really do not see Python being more of a candidate than C++ for displacing Fortran

by dsacco 9y ago

I have to be honest with you, as someone who writes in both C++ and Python, I really do not see Python being more of a candidate than C++ for displacing Fortran. Can you clarify why you think Python might be able to do it? For scientific computation with high performance requirements, Python is not competitive with Fortan or C++. For work that continues to happen in Fortran due to "academic inertia", my impression (and vague experience) is that researchers find the convenience of what they're accustomed to (Fortran) to be greater than the convenience of things like rapid prototyping offered by the various Python scientific computing and data analysis libraries. There is a mental overhead is switching, and I think most academics are sort of okay writing code in whatever is familiar and battle-tested if it means they can focus more on the research at hand. In other words, I'm not saying you're wrong, but I'm not following your reasoning.

vacri 9y ago
In a previous discussion of this kind (I'm not in this area myself), a scientist had said that the speed of the languages is less important than iterating the algorithms, and it was a lot easier for non-developer scientists to work with python than with C++.
cknight 9y ago
I worked as a programmer in a molecular dynamics research group for a while. I was asked to work in Python because it was what the boss was familiar with - so there's some of the same inertia happening again, just with a new(er) language, I guess. Speed is not a huge issue if you're happy to leave your simulation running overnight anyway, or if you have the option to just throw more and more cores at the problem (or in our case, both). The goal is to get your papers published. Code developing/interpreting/debugging time is generally far more serious an obstacle to that goal than simulation speed. I was the only developer there. Everyone else in the group coded on an almost daily basis, but none of them considered themselves programmers, and very few of them actually learn about good programming practices. They're biophysics researchers. Having not worked in Python much before I was also pretty pleased to find that, having determined that I needed to use Dijkstra's algorithm for path-finding and then working out the smallest Standard Deviation between certain sets of data points, Python came with libraries to do both of those things off the cuff. It's just so easy, I can see why it has a favoured place in this field.
- War_Machine 9y ago
  As long as Fortran is able to keep up with Python's capabilities (or any other language's for that matter), there seems to be absolutely no reason to make a change considering Fortran's history and familiarity in Physics. In other words, other languages need to make a big enough leap forward or Fortran to lag behind enough to justify a change. Your example just shows that it's fine to use other languages, and I agree, but it's not very compelling in promoting a change in language.
- hzhou321 9y ago
  > Speed is not a huge issue if you're happy to leave your simulation running overnight anyway, or if you have the option to just throw more and more cores at the problem What about those cases that it's the matter between throwing at 1000 core cluster and wether we can have results before next conference in a couple months. That is what really defines the scientific programming -- it's about feasible and infeasible. For other jobs, isn't it just matter of taste?
  - lostboys67 9y ago
    Depends not all scientific programming is at universities when I worked for BHR group (a Hydro dynamics research organisation) some times we had emergency projects that had < 24 hour turn around. I recall one where a clinet had had a serious issue at a plant and we ran the simulation and produced a report in a single day
harigov 9y ago
Just because python is slow doesn't necessarily mean that every library is also slow. With numpy/scipy, which are written in C with python bindings, scientific computation is pretty fast. Python is fast becoming standard language for data science with so many tools like notebooks that make it extremely easy to do many things. I am not sure how well it compares to Fortran but it is no brainer to use it over C++, especially for research needs.
- CardenB 9y ago
  Isn’t numpy written in Fortran?
  - dangerbird2 9y ago
    The native code python extensions for numpy are written in C, but numpy often links to matrix libraries written in Fortran like BLAS and LAPACK
  - Redoubts 9y ago
    Scipy is where a lot of the fortran is.
ptero 9y ago
A disclaimer -- I do not work in physics, however a couple of recent physics Ph.D. that I work with expressed similar views. Speed is not nearly as important now as it was 10-15 years ago. A typical scientist's worstation has 20 CPU cores. If I want to use 100 CPUs for a few days, it is trivial and 1000 is easy to get. Thus the fact that Python is slow(er) does not bother me unless I am setting up something major. What matters though, is the availability of libraries that let me reliably run my experiments. If there is a bug, it must be in my code, not the library -- a "discovery" caused by a software bug is humiliating. This is where Fortran shines and Python is not quite there yet -- the decades of beating on those libraries made them very well understood and trusted.
- DannyBee 9y ago
  "Speed is not nearly as important now as it was 10-15 years ago." Please try telling that to other users sometime. They do not seem to agree.
  - ptero 9y ago
    What users? I do work in research and this is what I see almost every time -- doubling the number of CPUs available for a simulation is trivial; getting someone to rewrite existing code to make it twice as fast on the same CPU is hard. I am not talking about interactive programs -- a slow browser is annoying. I am talking about scientific computing. This is just my experience, can you provide some counterexamples?
- lostboys67 9y ago
  So more cores means you can simulate to a higher resolution
brei 9y ago
I don't see Python displacing Fortran or C++, but I do see Python showing up in HPC as a platform for DSLs. Instead of handling the expensive part of the computation, Python acts as a UI layer / configuration manager / code generator. For an example, see fenics: [https://fenicsproject.org/ https://fenicsproject.org/]

by dsacco 9y ago

vacri 9y ago

In a previous discussion of this kind (I'm not in this area myself), a scientist had said that the speed of the languages is less important than iterating the algorithms, and it was a lot easier for non-developer scientists to work with python than with C++.

cknight 9y ago

I worked as a programmer in a molecular dynamics research group for a while. I was asked to work in Python because it was what the boss was familiar with - so there's some of the same inertia happening again, just with a new(er) language, I guess. Speed is not a huge issue if you're happy to leave your simulation running overnight anyway, or if you have the option to just throw more and more cores at the problem (or in our case, both). The goal is to get your papers published. Code developing/interpreting/debugging time is generally far more serious an obstacle to that goal than simulation speed. I was the only developer there. Everyone else in the group coded on an almost daily basis, but none of them considered themselves programmers, and very few of them actually learn about good programming practices. They're biophysics researchers. Having not worked in Python much before I was also pretty pleased to find that, having determined that I needed to use Dijkstra's algorithm for path-finding and then working out the smallest Standard Deviation between certain sets of data points, Python came with libraries to do both of those things off the cuff. It's just so easy, I can see why it has a favoured place in this field.

War_Machine 9y ago
As long as Fortran is able to keep up with Python's capabilities (or any other language's for that matter), there seems to be absolutely no reason to make a change considering Fortran's history and familiarity in Physics. In other words, other languages need to make a big enough leap forward or Fortran to lag behind enough to justify a change. Your example just shows that it's fine to use other languages, and I agree, but it's not very compelling in promoting a change in language.
hzhou321 9y ago
> Speed is not a huge issue if you're happy to leave your simulation running overnight anyway, or if you have the option to just throw more and more cores at the problem What about those cases that it's the matter between throwing at 1000 core cluster and wether we can have results before next conference in a couple months. That is what really defines the scientific programming -- it's about feasible and infeasible. For other jobs, isn't it just matter of taste?
- lostboys67 9y ago
  Depends not all scientific programming is at universities when I worked for BHR group (a Hydro dynamics research organisation) some times we had emergency projects that had < 24 hour turn around. I recall one where a clinet had had a serious issue at a plant and we ran the simulation and produced a report in a single day

harigov 9y ago

Just because python is slow doesn't necessarily mean that every library is also slow. With numpy/scipy, which are written in C with python bindings, scientific computation is pretty fast. Python is fast becoming standard language for data science with so many tools like notebooks that make it extremely easy to do many things. I am not sure how well it compares to Fortran but it is no brainer to use it over C++, especially for research needs.

CardenB 9y ago
Isn’t numpy written in Fortran?
- dangerbird2 9y ago
  The native code python extensions for numpy are written in C, but numpy often links to matrix libraries written in Fortran like BLAS and LAPACK
- Redoubts 9y ago
  Scipy is where a lot of the fortran is.

ptero 9y ago

A disclaimer -- I do not work in physics, however a couple of recent physics Ph.D. that I work with expressed similar views. Speed is not nearly as important now as it was 10-15 years ago. A typical scientist's worstation has 20 CPU cores. If I want to use 100 CPUs for a few days, it is trivial and 1000 is easy to get. Thus the fact that Python is slow(er) does not bother me unless I am setting up something major. What matters though, is the availability of libraries that let me reliably run my experiments. If there is a bug, it must be in my code, not the library -- a "discovery" caused by a software bug is humiliating. This is where Fortran shines and Python is not quite there yet -- the decades of beating on those libraries made them very well understood and trusted.

DannyBee 9y ago
"Speed is not nearly as important now as it was 10-15 years ago." Please try telling that to other users sometime. They do not seem to agree.
- ptero 9y ago
  What users? I do work in research and this is what I see almost every time -- doubling the number of CPUs available for a simulation is trivial; getting someone to rewrite existing code to make it twice as fast on the same CPU is hard. I am not talking about interactive programs -- a slow browser is annoying. I am talking about scientific computing. This is just my experience, can you provide some counterexamples?
lostboys67 9y ago
So more cores means you can simulate to a higher resolution

brei 9y ago

I don't see Python displacing Fortran or C++, but I do see Python showing up in HPC as a platform for DSLs. Instead of handling the expensive part of the computation, Python acts as a UI layer / configuration manager / code generator. For an example, see fenics: [https://fenicsproject.org/ https://fenicsproject.org/]