6 ms·

C, C++, Fortran are still used, most Python users just don't see it because it's hidden away underneath the calling function. I've been surprised by the rise o

C, C++, Fortran are still used, most Python users just don't see it because it's hidden away underneath the calling function. I've been surprised by the rise of Python in some ways although not at all in others. Languages like C, C++, Fortran, and dare I say it Rust are too low-level in their raw state for numerical computing. You had the US federal government funding language competitions because of this (see: Chapel). Languages like Python and R (and before that things like Lisp) came along and gave people a taste of something different, and it's obvious what people migrated to. Part of it is timing: multivariate computational statistics (ML/data science/DL/whatever you want to call it) just sort of started taking off in computer science communities before LLVM-based languages like Julia or Nim could get a foothold. OCaml might have fit that niche but never got there because of a desire to take a different path, or take the path more slowly. So people looked for a nice expressive language, found it in Python, and buried all the messy stuff behind wrapper functions and called it a day. It was furthered along by Matlab being another comparison on the other side -- Python looks kludgy compared to modern Fortran or C, but not compared to Matlab. All that wrapper time in Python has its costs, so I suspect as limits get pushed further we'll eventually see a migration to something else like Julia or Nim, or something else not on anyone's radar. One moral to this story is that expressiveness matters. People will go out of their way to avoid talking directly to machines at a low level.

zozbot234 5y ago
> You had the US federal government funding language competitions because of this (see: Chapel). Wait, weren't they supposed to be using Ada for everything anyway? What's wrong with Ada?
- tormeh 5y ago
  Different design objectives, I believe.
Robotbeat 5y ago
It’s because Matlab (and Mathematica, etc) is proprietary, and therefore you always have to pay the Danegeld. So we use numpy instead because it’s extensible, it uses all the super fast C/C++/FORTRAN stuff on the backend, and is fairly easy to learn. I actually still would prefer Matlab as the syntax is more compact and natural than numpy (which is like a matlabified Python), but that’s probably just due to more experience in Matlab.
- pm90 5y ago
  Yep. When I was in grad school all the labs were furiously migrating away from matlab because of its costs and confusing licensing around running multiple replicas.
- kragen 5y ago
  Octave is free software with Matlab syntax and Matlab-style interactivity (autoreload, etc.) I'm not a huge fan of the language (Matlab/Octave) but it certainly does make it quick to whip things up.
  - Robotbeat 5y ago
    It sucks compared to Matlab, though. Unfortunately. (Sci lab is better, altho not compatible.) But I have also used it in a pinch. Size of the community means Matlab or Numpy are your best options. If you aren’t happy with Matlab due to cost or licensing stuff, numpy is really good. Also integrates with a lot of Python stuff like machine vision, machine learning, etc, which have expensive or nonexistent packages in Matlab.
    - kragen 5y ago
      Interesting! What are the drawbacks of Octave?
      - sideshowb 5y ago
        It feels like MATLAB? ;-)
        kragen 5y ago
        I mean relative to Matlab :)
      - ravel-bar-foo 5y ago
        I used Octave for a year when my institution's Matlab license servers were being improperly administered. (I had a lot of project code written in Matlab, but the license server going down on the weekend before a conference deadline with nobody available to reboot it until Monday was a dealbreaker.) The biggest stumbling block was that Matlab has a huge and heavily used proprietary package library, and a lot of my existing code, (official) tutorials and Stack Overflow code assumed these libraries were available. In Octave I found myself reimplementing the newer parts of the Matlab image processing libraries. This led to the discovery that the Matlab and Octave builtins for handling image data are subtly different, so I ended up having to run tests in the code and write different conditional flows to make it cross-compatible. There are also subtle differences in basic behaviors (was it variable scoping? file handling?) which resulted in some surprise and frustration. Following the licensing and Octave debacle, all my latest code is written in numpy.
nneonneo 5y ago
> Python looks kludgy compared to modern Fortran or C I’m not sure I can agree with this. Both Python and Matlab provide very nice, high level ways to interact with multidimensional data using simple syntax. Under the hood, both will wind up using fast algorithms to implement the operations. C and Fortran require much more low-level considerations like manually managing memory, futzing with pointers or indices, and generally writing a lot more boilerplate code to shuffle data around. Matlab, despite all its quirks, could probably have won if it was open source. It’s got a very long history of use in scientific computation and a large user base despite its high price.
- auxym 5y ago
  Matlab works fine for anything purely "numerical" but fails hard as soon as you need to do more "general computing". Just string handling for example. Or, as far as I know, it's still not possible to implement a custom CLI interface in a matlab script, like you would with argparse in python. Matlab also historically was really bad for abstraction and code architecture in general. For example, the hard "1 function per file" rule, which encouraged people to not use functions at all, or if you really had to, write 2 or 3 really huge functions (in separate files). Only in recent years (the past 5 or 10 years) did matlab get OOP stuff (classes) and the option for multiple (private) functions in a single script file (still only one public/exported function is possible per file, because the file name is the function name and matlab uses path-based resolution).
- leephillips 5y ago
  Fortran does not require (nor has much available for) manual memory management, and its array syntax is more convenient than Numpy (and far more convenient than Python without Numpy), obviating any futzing around with pointers or indices.
  - pletnes 5y ago
    Fortran is definitely much more high level than C, and way easier to write performant numerics in, than the C family languages. I don’t follow why its array syntax is easier than python though. They’re mostly very similar and the numpy developers seem to come from a Fortran background.
cb321 5y ago
You may know this, but since you always mentioned Nim & Julia together, it might confuse passers by. Nim does not, in fact, need LLVM (though there is a hobby side project using that). Mainline Nim compiles directly to C (or C++ or Javascript) and people even use it on embedded systems. What seems to attract scientists is the REPL and/or notebook UI style/focus of Matlab/Mathematica/Python/Julia/R/... As projects migrate from exploratory to production, optimizing for interactivity becomes a burden -- whether it is Julia Time To First Plots or dynamic typing causing performance and stability/correctness problems in Python code or even just more careful unit tests. They are just very different mindsets - "show me an answer pronto" vs. "more care". "Gradually typed" systems like Cython or Common Lisp's `declare` can sometimes ease the transition, but often it's a lot of work to move code from everything-is-a-generic-object to articulated types, and often exploratory code written by scientists is...really rough proof of concept stuff.
- nextos 5y ago
  The time to first plots in Julia is drastically lower now. And still, it was something you only paid once per session, due to JIT. Julia is the first language I find truly pleasant to use in this domain. I am more than happy to pay a small initial JIT overhead in exchange for code that looks like Ruby but runs 1/2 the speed of decent C++. Plus, lots of libraries are really high quality and composable. Python has exceptionally good libraries, but they tend to be big monoliths. This makes me feel Julia or something like Julia will win in the long run.
  - exdsq 5y ago
    Julia runs 2x the speed of decent C++?!
    - nextos 5y ago
      Sorry I meant 1/2 the speed or 2x the time, edited :) Consider that BLAS written in pure Julia has very decent performance. If you are into numerical computing, you will quickly understand this is crazy. Carefully written Julia tends to be surprisingly fast. Excessive allocations tend to be a bigger performance problem than raw speed. Of course excessive allocations eventually have an impact on speed as well. There are some idiomatic ways to avoid this.
kzrdude 5y ago
Python was pragmatic and adopted changes that numpy needed and advocated for. Maybe Julia is the only other worthy comparison? Also, dynamic typing is a boon - and default & keyword arguments is a great feature for complicated, versatile, useful algorithm implementations and interfaces to them. Both of these features have a cost in bigger programs, but they really make Python stand out.
_dain_ 5y ago
Nim is not LLVM based, it compiles to plain old C/C++.
- adenozine 5y ago
  Nim actually has LLVM support via nlvm https://github.com/arnetheduck/nlvm https://github.com/arnetheduck/nlvm It's not officially blessed, but it does work.
ip26 5y ago
People will go out of their way to avoid talking directly to machines at a low level I would put it differently. At 30 bugs per kLOC, I'd prefer my codebase expresses a problem & it's solution- and as little below that level as possible. Each well-vetted layer of abstraction between a scientific programmer and the machine's low level interface eliminates whole classes of bugs that are irrelevant to the problem that user is actually working on.
- DasIch 5y ago
  > At 30 bugs per kLOC[...] Where does that number come from?
  - arthurcolle 5y ago
    The context suggests to me that this was a self-reported approximation from GP
  - ip26 5y ago
    It’s a median-ish of various studies. You can google “bugs per 1000 lines of code”. The important part wasn’t the exact number, but rather the discovery that the ratio is pretty stable.
    - marcosdumay 5y ago
      > but rather the discovery that the ratio is pretty stable The thing is, it isn't stable. It just doesn't depend on the language, what is very surprising. But it varies enormously from one study to another and, AFAIK, nobody has a good set of factors explaining it.
      - fauigerzigerk 5y ago
        I don't find it that surprising. I think what programming languages (and styles) do is fill up each line of code with information until a roughly constant level of cognitive effort is required to process that line. At that constant level of effort, we make a certain constant number of mistakes. And that's what I think these studies show.
pdonis 5y ago
> C, C++, Fortran are still used, most Python users just don't see it because it's hidden away underneath the calling function. Yes, the article talks about this: Python is a glue language and the actual heavy duty computation is being done inside an extension module like numpy that's written in a faster language.
- joshuamorton 5y ago
  OpenBLAS and LAPACK are mostly Fortran and numpy will use them if present on the system.
- jurschreuder 5y ago
  A single threaded, not vectorized for loop in c++ runs faster than calling BLAS from Python with Numpy though. The Python glue makes everything slow. For example if you read out the camera in Python using OpenCV you'll get a way lower framerate than if you do the same from c++, even if in both cases you use OpenCV which is c++.
agumonkey 5y ago
Isn't it the usual "dynlang as prototyping clay" story ? python (with FFI -> native libs) gets you iterate over ideas faster and leaner.
michaelsbradley 5y ago
before LLVM-based languages like Julia or Nim could get a foothold. Nim isn't an LLVM-based language (not 100% sure you were implying that), though nlvm[1] is a thing. Regardless, Arraymancer[2] is quite a gem in the scientific computing ecosystem (doesn't build on nlvm, i.e. the mainline Nim compiler is sufficient). [1] https://github.com/arnetheduck/nlvm https://github.com/arnetheduck/nlvm [2] https://github.com/mratsim/Arraymancer https://github.com/mratsim/Arraymancer

zozbot234 5y ago

> You had the US federal government funding language competitions because of this (see: Chapel). Wait, weren't they supposed to be using Ada for everything anyway? What's wrong with Ada?

tormeh 5y ago
Different design objectives, I believe.

Robotbeat 5y ago

It’s because Matlab (and Mathematica, etc) is proprietary, and therefore you always have to pay the Danegeld. So we use numpy instead because it’s extensible, it uses all the super fast C/C++/FORTRAN stuff on the backend, and is fairly easy to learn. I actually still would prefer Matlab as the syntax is more compact and natural than numpy (which is like a matlabified Python), but that’s probably just due to more experience in Matlab.

pm90 5y ago
Yep. When I was in grad school all the labs were furiously migrating away from matlab because of its costs and confusing licensing around running multiple replicas.
kragen 5y ago
Octave is free software with Matlab syntax and Matlab-style interactivity (autoreload, etc.) I'm not a huge fan of the language (Matlab/Octave) but it certainly does make it quick to whip things up.
- Robotbeat 5y ago
  It sucks compared to Matlab, though. Unfortunately. (Sci lab is better, altho not compatible.) But I have also used it in a pinch. Size of the community means Matlab or Numpy are your best options. If you aren’t happy with Matlab due to cost or licensing stuff, numpy is really good. Also integrates with a lot of Python stuff like machine vision, machine learning, etc, which have expensive or nonexistent packages in Matlab.
  - kragen 5y ago
    Interesting! What are the drawbacks of Octave?
    - sideshowb 5y ago
      It feels like MATLAB? ;-)
      - kragen 5y ago
        I mean relative to Matlab :)
    - ravel-bar-foo 5y ago
      I used Octave for a year when my institution's Matlab license servers were being improperly administered. (I had a lot of project code written in Matlab, but the license server going down on the weekend before a conference deadline with nobody available to reboot it until Monday was a dealbreaker.) The biggest stumbling block was that Matlab has a huge and heavily used proprietary package library, and a lot of my existing code, (official) tutorials and Stack Overflow code assumed these libraries were available. In Octave I found myself reimplementing the newer parts of the Matlab image processing libraries. This led to the discovery that the Matlab and Octave builtins for handling image data are subtly different, so I ended up having to run tests in the code and write different conditional flows to make it cross-compatible. There are also subtle differences in basic behaviors (was it variable scoping? file handling?) which resulted in some surprise and frustration. Following the licensing and Octave debacle, all my latest code is written in numpy.

nneonneo 5y ago

> Python looks kludgy compared to modern Fortran or C I’m not sure I can agree with this. Both Python and Matlab provide very nice, high level ways to interact with multidimensional data using simple syntax. Under the hood, both will wind up using fast algorithms to implement the operations. C and Fortran require much more low-level considerations like manually managing memory, futzing with pointers or indices, and generally writing a lot more boilerplate code to shuffle data around. Matlab, despite all its quirks, could probably have won if it was open source. It’s got a very long history of use in scientific computation and a large user base despite its high price.

auxym 5y ago
Matlab works fine for anything purely "numerical" but fails hard as soon as you need to do more "general computing". Just string handling for example. Or, as far as I know, it's still not possible to implement a custom CLI interface in a matlab script, like you would with argparse in python. Matlab also historically was really bad for abstraction and code architecture in general. For example, the hard "1 function per file" rule, which encouraged people to not use functions at all, or if you really had to, write 2 or 3 really huge functions (in separate files). Only in recent years (the past 5 or 10 years) did matlab get OOP stuff (classes) and the option for multiple (private) functions in a single script file (still only one public/exported function is possible per file, because the file name is the function name and matlab uses path-based resolution).
leephillips 5y ago
Fortran does not require (nor has much available for) manual memory management, and its array syntax is more convenient than Numpy (and far more convenient than Python without Numpy), obviating any futzing around with pointers or indices.
- pletnes 5y ago
  Fortran is definitely much more high level than C, and way easier to write performant numerics in, than the C family languages. I don’t follow why its array syntax is easier than python though. They’re mostly very similar and the numpy developers seem to come from a Fortran background.

cb321 5y ago

You may know this, but since you always mentioned Nim & Julia together, it might confuse passers by. Nim does not, in fact, need LLVM (though there is a hobby side project using that). Mainline Nim compiles directly to C (or C++ or Javascript) and people even use it on embedded systems. What seems to attract scientists is the REPL and/or notebook UI style/focus of Matlab/Mathematica/Python/Julia/R/... As projects migrate from exploratory to production, optimizing for interactivity becomes a burden -- whether it is Julia Time To First Plots or dynamic typing causing performance and stability/correctness problems in Python code or even just more careful unit tests. They are just very different mindsets - "show me an answer pronto" vs. "more care". "Gradually typed" systems like Cython or Common Lisp's `declare` can sometimes ease the transition, but often it's a lot of work to move code from everything-is-a-generic-object to articulated types, and often exploratory code written by scientists is...really rough proof of concept stuff.

nextos 5y ago
The time to first plots in Julia is drastically lower now. And still, it was something you only paid once per session, due to JIT. Julia is the first language I find truly pleasant to use in this domain. I am more than happy to pay a small initial JIT overhead in exchange for code that looks like Ruby but runs 1/2 the speed of decent C++. Plus, lots of libraries are really high quality and composable. Python has exceptionally good libraries, but they tend to be big monoliths. This makes me feel Julia or something like Julia will win in the long run.
- exdsq 5y ago
  Julia runs 2x the speed of decent C++?!
  - nextos 5y ago
    Sorry I meant 1/2 the speed or 2x the time, edited :) Consider that BLAS written in pure Julia has very decent performance. If you are into numerical computing, you will quickly understand this is crazy. Carefully written Julia tends to be surprisingly fast. Excessive allocations tend to be a bigger performance problem than raw speed. Of course excessive allocations eventually have an impact on speed as well. There are some idiomatic ways to avoid this.

kzrdude 5y ago

Python was pragmatic and adopted changes that numpy needed and advocated for. Maybe Julia is the only other worthy comparison? Also, dynamic typing is a boon - and default & keyword arguments is a great feature for complicated, versatile, useful algorithm implementations and interfaces to them. Both of these features have a cost in bigger programs, but they really make Python stand out.

_dain_ 5y ago

Nim is not LLVM based, it compiles to plain old C/C++.

adenozine 5y ago
Nim actually has LLVM support via nlvm https://github.com/arnetheduck/nlvm https://github.com/arnetheduck/nlvm It's not officially blessed, but it does work.

ip26 5y ago

People will go out of their way to avoid talking directly to machines at a low level I would put it differently. At 30 bugs per kLOC, I'd prefer my codebase expresses a problem & it's solution- and as little below that level as possible. Each well-vetted layer of abstraction between a scientific programmer and the machine's low level interface eliminates whole classes of bugs that are irrelevant to the problem that user is actually working on.

DasIch 5y ago
> At 30 bugs per kLOC[...] Where does that number come from?
- arthurcolle 5y ago
  The context suggests to me that this was a self-reported approximation from GP
- ip26 5y ago
  It’s a median-ish of various studies. You can google “bugs per 1000 lines of code”. The important part wasn’t the exact number, but rather the discovery that the ratio is pretty stable.
  - marcosdumay 5y ago
    > but rather the discovery that the ratio is pretty stable The thing is, it isn't stable. It just doesn't depend on the language, what is very surprising. But it varies enormously from one study to another and, AFAIK, nobody has a good set of factors explaining it.
    - fauigerzigerk 5y ago
      I don't find it that surprising. I think what programming languages (and styles) do is fill up each line of code with information until a roughly constant level of cognitive effort is required to process that line. At that constant level of effort, we make a certain constant number of mistakes. And that's what I think these studies show.

pdonis 5y ago

> C, C++, Fortran are still used, most Python users just don't see it because it's hidden away underneath the calling function. Yes, the article talks about this: Python is a glue language and the actual heavy duty computation is being done inside an extension module like numpy that's written in a faster language.

joshuamorton 5y ago
OpenBLAS and LAPACK are mostly Fortran and numpy will use them if present on the system.
jurschreuder 5y ago
A single threaded, not vectorized for loop in c++ runs faster than calling BLAS from Python with Numpy though. The Python glue makes everything slow. For example if you read out the camera in Python using OpenCV you'll get a way lower framerate than if you do the same from c++, even if in both cases you use OpenCV which is c++.

agumonkey 5y ago

Isn't it the usual "dynlang as prototyping clay" story ? python (with FFI -> native libs) gets you iterate over ideas faster and leaner.

michaelsbradley 5y ago

before LLVM-based languages like Julia or Nim could get a foothold. Nim isn't an LLVM-based language (not 100% sure you were implying that), though nlvm[1] is a thing. Regardless, Arraymancer[2] is quite a gem in the scientific computing ecosystem (doesn't build on nlvm, i.e. the mainline Nim compiler is sufficient). [1] https://github.com/arnetheduck/nlvm https://github.com/arnetheduck/nlvm [2] https://github.com/mratsim/Arraymancer https://github.com/mratsim/Arraymancer