7 ms·

I agree with you. However, note that many people who are using Python for writing scientific code make use of C/C++ in one way or the other (aside from NumPy, S

by rasbt 11y ago

I agree with you. However, note that many people who are using Python for writing scientific code make use of C/C++ in one way or the other (aside from NumPy, SciPy, and Theano). For example, many people write the "most intensive" computations down in C/C++/Cython if they call those functions frequently -- Python becomes a wrapper. One example that pops into my mind is khmer (https://github.com/dib-lab/khmer https://github.com/dib-lab/khmer)

RA_Fisher 11y ago
I believe S (ancestor of R) started as C glue at Bell Labs. I've heard it said a few times that R is slow, but that doesn't really make much sense if your bottleneck routines are R calls to C++.
srean 11y ago
This is a common refrain: drop down to C, C++, Fortran for the computation intensive parts. It works, but only to a degree. The inefficiencies lie in the vectorization semantics of the host language(s) that leads to extra copies and extra levels of indirection. So this dual language mode of operation typically does not approach what one could have obtained had one disposed the baggage entirely, except for I/O. Usually in the quest for better speed, that is what remains, as one moves progressively larger portions of the application in the C, C++, Fortran part of the code. A reason I like Julia is that I can largely avoid this dual language annoyance, and enjoy the succinctness of pithy vectorized expressions using https://github.com/lindahua/Devectorize.jl https://github.com/lindahua/Devectorize.jl
- syllogism 11y ago
  You don't have to use the vectorization though. I prefer to work with Cython and just use pointers. You don't get to write vector + vector --- you have to actually write the loop --- but that's not such a big deal imo.
  - srean 11y ago
    I am lazy and super prone to make one-off errors, so that to me is the _biggest_deal_ TM :)
- deleted 11y ago
  [deleted]
- Lofkin 11y ago
  Numba for python also has a no copies vectorization mode.
- nxb 11y ago
  The speed performance of C/C++ is terrible, compared to GPU speed, which is often 50x to 100x faster. The goal has shifted and continues to shift in that direction. C/C++ is no longer best for speed, not even close. Now, all that matters is which language has the libraries that make it easiest to get custom code onto the GPU. Python and Lua seem to be winning there, by far.
  - leni536 11y ago
    >Now, all that matters is which library makes it easiest to get custom code onto the GPU. Python and Lua seem to be winning there, by far. This is interesting. How is it possible that python and lua have more efficient wrappers around GPU libraries? Also there are many GPU libraries for C/C++ too. Armadillo can use NVBLAS as a backend too. I'm not sure if I get your point of C/C++ being slow.
    - nxb 11y ago
      It's not about wrapping. The real power is in the cross-compilation of expressions and entire complex data pipelines, from a simple-as-possible high-level language into GPU language. That's the power at the core of, e.g. Theano.
      - srean 11y ago
        Compiling high level instructions to different hardware backends is hardly an exclusive feature of python and lua libraries. Google would swamp you with hits if you were to search
        nxb 11y ago
        Show me one C/C++ library that competes with Theano or Torch7? Google / Facebook and many other huge companies are using Theano and Torch7 in production, at scale. The ML industry has been continuously moving in this direction for years now. On these optimized ML systems, only a tiny fraction of CPU time is spent outside of the GPU. The goal in many of these companies is to migrate all tasks that can be done on GPUs to GPUs, as soon as possible. It's far faster and more cost efficient.

by rasbt 11y ago

RA_Fisher 11y ago

I believe S (ancestor of R) started as C glue at Bell Labs. I've heard it said a few times that R is slow, but that doesn't really make much sense if your bottleneck routines are R calls to C++.

srean 11y ago

This is a common refrain: drop down to C, C++, Fortran for the computation intensive parts. It works, but only to a degree. The inefficiencies lie in the vectorization semantics of the host language(s) that leads to extra copies and extra levels of indirection. So this dual language mode of operation typically does not approach what one could have obtained had one disposed the baggage entirely, except for I/O. Usually in the quest for better speed, that is what remains, as one moves progressively larger portions of the application in the C, C++, Fortran part of the code. A reason I like Julia is that I can largely avoid this dual language annoyance, and enjoy the succinctness of pithy vectorized expressions using https://github.com/lindahua/Devectorize.jl https://github.com/lindahua/Devectorize.jl

syllogism 11y ago
You don't have to use the vectorization though. I prefer to work with Cython and just use pointers. You don't get to write vector + vector --- you have to actually write the loop --- but that's not such a big deal imo.
- srean 11y ago
  I am lazy and super prone to make one-off errors, so that to me is the _biggest_deal_ TM :)
deleted 11y ago
[deleted]
Lofkin 11y ago
Numba for python also has a no copies vectorization mode.
nxb 11y ago
The speed performance of C/C++ is terrible, compared to GPU speed, which is often 50x to 100x faster. The goal has shifted and continues to shift in that direction. C/C++ is no longer best for speed, not even close. Now, all that matters is which language has the libraries that make it easiest to get custom code onto the GPU. Python and Lua seem to be winning there, by far.
- leni536 11y ago
  >Now, all that matters is which library makes it easiest to get custom code onto the GPU. Python and Lua seem to be winning there, by far. This is interesting. How is it possible that python and lua have more efficient wrappers around GPU libraries? Also there are many GPU libraries for C/C++ too. Armadillo can use NVBLAS as a backend too. I'm not sure if I get your point of C/C++ being slow.
  - nxb 11y ago
    It's not about wrapping. The real power is in the cross-compilation of expressions and entire complex data pipelines, from a simple-as-possible high-level language into GPU language. That's the power at the core of, e.g. Theano.
    - srean 11y ago
      Compiling high level instructions to different hardware backends is hardly an exclusive feature of python and lua libraries. Google would swamp you with hits if you were to search
      - nxb 11y ago
        Show me one C/C++ library that competes with Theano or Torch7? Google / Facebook and many other huge companies are using Theano and Torch7 in production, at scale. The ML industry has been continuously moving in this direction for years now. On these optimized ML systems, only a tiny fraction of CPU time is spent outside of the GPU. The goal in many of these companies is to migrate all tasks that can be done on GPUs to GPUs, as soon as possible. It's far faster and more cost efficient.