4 ms·

I doubt, that you will find it easy to beat well written numeric Python code with naive C, C++ or Fortran approach. Python is not inherently slower than C, C++

I doubt, that you will find it easy to beat well written numeric Python code with naive C, C++ or Fortran approach. Python is not inherently slower than C, C++ or Fortran. It is just a language after all. If you have a fast implementation of computational engine, like Numpy, you can reach speeds of C++ in computational tasks. If you have an optimizer that simplifies your math expressions, before running them - you can get better than naive C++ approach. An optimizer and computational engine that optimally offloads work from CPU to GPU can give an order of magnitude advantage over naive C++/blas code. And to give you a concrete example - there is a nice Python library called Theano, that is doing just than.

hazov 14y ago
Numpy uses Lapack, no? With vectorization and compiler optimizations* I doubt pure Python, well CPython running pure Python, would match Fortran and some C open source solvers that are the state of art like Lapack for dense matrices and superlu or plastix for sparse ones. It's not much because of the language, but mainly because python has more overhead than C or Fortran. Maybe as you said python can work well in solving a single linear system using GPU. I worked with this before GPUs came to be used for that, so I can't comment on this one. * Generally compilers optimize mathematical expressions unless they involve complicated pointer arithmetic, this is why Fortran is still used in this area, Fortran does not have this feature.
- freyrs3 14y ago
  In numeric Python packages computations are never done inside the CPython runtime, they're done in specialized kernels written in C, Fortran and sometimes fine-tuned assembly including a sampling of the various BLAS/LAPACK flavors... so yes NumPy can often be competitive with or outperform libraries written in lower-level languages.
sliverstorm 14y ago
You assume all the CFD guys are writing shitty code. It's not like Python is a prerequisite to writing good code.
thomasjames 14y ago
Yup, just gonna fire up an interpreted language on the cluster and see how my other buddies at the ole national lab react.
- cdavid 14y ago
  Lots of people do just that, e.g. http://www.training.prace-ri.eu/uploads/tx_pracetmo/pythonHPC.pdf http://www.training.prace-ri.eu/uploads/tx_pracetmo/pythonHP... Maybe you should change buddies
  - thomasjames 14y ago
    So you still link to functions in compiled language code to do the actual leg work and all threading has to be done in C extensions, or am I missing something?
    - cdavid 14y ago
      yes you are. NumPy is much more than a wrapper around C libraries - if it were just that, it would exist in most languages, and yet few general programming languages have something like numpy. The whole point is that it gives you abstractions on top of those libraries, like broadcasting, ufunc, fancy indexing, etc...
coldtea 14y ago
>I doubt, that you will find it easy to beat well written numeric Python code with naive C, C++ or Fortran approach. Huh? People do it all the time. Naive, but not memory-leaking or improper complexity using, C/C++ etc code, beats Python hands town -- and can be more than 10-20 times faster. >Python is not inherently slower than C, C++ or Fortran. It is just a language after all. An interpreted language, with a not-that-good interpreter and garbage collector, non primitive integers and other such things holding it back. >If you have a fast implementation of computational engine, like Numpy, you can reach speeds of C++ in computational tasks. That's because the "fast implementation" is NOT written in Python.
- dchichkov 14y ago
  Let's go down to some example and consider a typical numeric problem, for example estimation of a cross entropy gradient of some function on your data. With Python (and Theano computational engine) you can write something along the lines: def cost(goal, prediction): crossEntropy = -goal * log(prediction) - (1 - goal) * log(goal - prediction) return mean(crossEntropy) prediction = 1 / (1 + exp( .... -dot(x,w) - b + ...)) > 0.5 gradW, gradB = grad(cost(y, prediction) + (w**2).sum(), [w,b]) And then just apply that function to a matrix containing your data. That's it. When you apply a function, it will be interpreted, converted into a computation graph, this computation graph will be optimized and parts of the computation will be offloaded to GPU with memory transfers between the host and GPU taken care of, and you will get a result in a user friendly and efficient Numpy array. The resulting computation will be nearly optimal and limited by memory bandwith, CPU - GPU bus bandwidth and GPU FOPS rate. With luck you can get close to theoretical maximum of your GPU floating point performance. And all done in a few lines of Python. Now consider the same in C++. Yes, it can be done. But there are just no open source libraries available that can do that. Closest open-source implementation that I know of is gpumatrix, a port of C++ Eigen library to GPU. And it doesn't even come close to what is available in Python. So with C++, if you want to match the performance of these few lines of Python code, good luck studying Cuda or OpenCL and implementing the computation engine right, from the first time. (disclaimer) I'm not in any way affiliated with OP and I actually use (and like) C/C++ a lot.

hazov 14y ago

Numpy uses Lapack, no? With vectorization and compiler optimizations* I doubt pure Python, well CPython running pure Python, would match Fortran and some C open source solvers that are the state of art like Lapack for dense matrices and superlu or plastix for sparse ones. It's not much because of the language, but mainly because python has more overhead than C or Fortran. Maybe as you said python can work well in solving a single linear system using GPU. I worked with this before GPUs came to be used for that, so I can't comment on this one. * Generally compilers optimize mathematical expressions unless they involve complicated pointer arithmetic, this is why Fortran is still used in this area, Fortran does not have this feature.

freyrs3 14y ago
In numeric Python packages computations are never done inside the CPython runtime, they're done in specialized kernels written in C, Fortran and sometimes fine-tuned assembly including a sampling of the various BLAS/LAPACK flavors... so yes NumPy can often be competitive with or outperform libraries written in lower-level languages.

sliverstorm 14y ago

You assume all the CFD guys are writing shitty code. It's not like Python is a prerequisite to writing good code.

thomasjames 14y ago

Yup, just gonna fire up an interpreted language on the cluster and see how my other buddies at the ole national lab react.

cdavid 14y ago
Lots of people do just that, e.g. http://www.training.prace-ri.eu/uploads/tx_pracetmo/pythonHPC.pdf http://www.training.prace-ri.eu/uploads/tx_pracetmo/pythonHP... Maybe you should change buddies
- thomasjames 14y ago
  So you still link to functions in compiled language code to do the actual leg work and all threading has to be done in C extensions, or am I missing something?
  - cdavid 14y ago
    yes you are. NumPy is much more than a wrapper around C libraries - if it were just that, it would exist in most languages, and yet few general programming languages have something like numpy. The whole point is that it gives you abstractions on top of those libraries, like broadcasting, ufunc, fancy indexing, etc...

coldtea 14y ago

>I doubt, that you will find it easy to beat well written numeric Python code with naive C, C++ or Fortran approach. Huh? People do it all the time. Naive, but not memory-leaking or improper complexity using, C/C++ etc code, beats Python hands town -- and can be more than 10-20 times faster. >Python is not inherently slower than C, C++ or Fortran. It is just a language after all. An interpreted language, with a not-that-good interpreter and garbage collector, non primitive integers and other such things holding it back. >If you have a fast implementation of computational engine, like Numpy, you can reach speeds of C++ in computational tasks. That's because the "fast implementation" is NOT written in Python.

dchichkov 14y ago
Let's go down to some example and consider a typical numeric problem, for example estimation of a cross entropy gradient of some function on your data. With Python (and Theano computational engine) you can write something along the lines: def cost(goal, prediction): crossEntropy = -goal * log(prediction) - (1 - goal) * log(goal - prediction) return mean(crossEntropy) prediction = 1 / (1 + exp( .... -dot(x,w) - b + ...)) > 0.5 gradW, gradB = grad(cost(y, prediction) + (w**2).sum(), [w,b]) And then just apply that function to a matrix containing your data. That's it. When you apply a function, it will be interpreted, converted into a computation graph, this computation graph will be optimized and parts of the computation will be offloaded to GPU with memory transfers between the host and GPU taken care of, and you will get a result in a user friendly and efficient Numpy array. The resulting computation will be nearly optimal and limited by memory bandwith, CPU - GPU bus bandwidth and GPU FOPS rate. With luck you can get close to theoretical maximum of your GPU floating point performance. And all done in a few lines of Python. Now consider the same in C++. Yes, it can be done. But there are just no open source libraries available that can do that. Closest open-source implementation that I know of is gpumatrix, a port of C++ Eigen library to GPU. And it doesn't even come close to what is available in Python. So with C++, if you want to match the performance of these few lines of Python code, good luck studying Cuda or OpenCL and implementing the computation engine right, from the first time. (disclaimer) I'm not in any way affiliated with OP and I actually use (and like) C/C++ a lot.