5 ms·

And there is nothing wrong with C++. For linear algebra I use the armadillo library and it's really a nice wrapper around LAPACK and BLAS (and fast!). For some

by leni536 11y ago

And there is nothing wrong with C++. For linear algebra I use the armadillo library and it's really a nice wrapper around LAPACK and BLAS (and fast!). For some reason scientists are somewhat afraid of C++. For some reason you "have to" prototype in an "easier" language. Sure, you can't use C++ as a calculator as opposed to interpreted languages, but I see people being stuck with their computations at the prototyping language and eventually not bringing it to a faster platform. Point being: C++ is not hard for scientific calculations.

rasbt 11y ago
I agree with you. However, note that many people who are using Python for writing scientific code make use of C/C++ in one way or the other (aside from NumPy, SciPy, and Theano). For example, many people write the "most intensive" computations down in C/C++/Cython if they call those functions frequently -- Python becomes a wrapper. One example that pops into my mind is khmer (https://github.com/dib-lab/khmer https://github.com/dib-lab/khmer)
- RA_Fisher 11y ago
  I believe S (ancestor of R) started as C glue at Bell Labs. I've heard it said a few times that R is slow, but that doesn't really make much sense if your bottleneck routines are R calls to C++.
- srean 11y ago
  This is a common refrain: drop down to C, C++, Fortran for the computation intensive parts. It works, but only to a degree. The inefficiencies lie in the vectorization semantics of the host language(s) that leads to extra copies and extra levels of indirection. So this dual language mode of operation typically does not approach what one could have obtained had one disposed the baggage entirely, except for I/O. Usually in the quest for better speed, that is what remains, as one moves progressively larger portions of the application in the C, C++, Fortran part of the code. A reason I like Julia is that I can largely avoid this dual language annoyance, and enjoy the succinctness of pithy vectorized expressions using https://github.com/lindahua/Devectorize.jl https://github.com/lindahua/Devectorize.jl
  - syllogism 11y ago
    You don't have to use the vectorization though. I prefer to work with Cython and just use pointers. You don't get to write vector + vector --- you have to actually write the loop --- but that's not such a big deal imo.
    - srean 11y ago
      I am lazy and super prone to make one-off errors, so that to me is the _biggest_deal_ TM :)
  - deleted 11y ago
    [deleted]
  - Lofkin 11y ago
    Numba for python also has a no copies vectorization mode.
  - nxb 11y ago
    The speed performance of C/C++ is terrible, compared to GPU speed, which is often 50x to 100x faster. The goal has shifted and continues to shift in that direction. C/C++ is no longer best for speed, not even close. Now, all that matters is which language has the libraries that make it easiest to get custom code onto the GPU. Python and Lua seem to be winning there, by far.
    - leni536 11y ago
      >Now, all that matters is which library makes it easiest to get custom code onto the GPU. Python and Lua seem to be winning there, by far. This is interesting. How is it possible that python and lua have more efficient wrappers around GPU libraries? Also there are many GPU libraries for C/C++ too. Armadillo can use NVBLAS as a backend too. I'm not sure if I get your point of C/C++ being slow.
      - nxb 11y ago
        It's not about wrapping. The real power is in the cross-compilation of expressions and entire complex data pipelines, from a simple-as-possible high-level language into GPU language. That's the power at the core of, e.g. Theano.
cinquemb 11y ago
I see this a lot in my lab (days/weeks to run computations and analysis that could take minutes/hours), and I used to offer to help the postdocs/phd students port their code from matlab to C/C++, but I've mostly given up on offering unless they ask for help, or if it seems like fun. I also think it's because they mostly think they won't work on these things again (which is kind of weird to think about since they spent most of their life working to get to this point, but that's another conversation). Aside, I think armadillo is pretty great, especially going back and forth with other matrix libraries, also nice wrapper around OpenBLAS which made not running something on a cluster more bearable.
srean 11y ago
I saw you mention Armadillo a few times, seems you are having a lot of fun with it. Another somewhat look-alike is Eigen, its pretty nice too. These are all modern takes on the original sin Blitz++ which is actually more full featured in what they can represent than Eigen, Armadillo, mublas (by Boost) and Blaze: it supports multidimensional arrays as opposed to just (2d) matrices. Where Blitz++ falls short of the competition is full use of SIMD instructions. However, G++ does a plenty good job of vectorization, but the king of the hill is still Intel's compiler.
- danieldk 11y ago
  I am not sure about Intel's compiler, but OpenBLAS is pretty much on par with Intel MKL in most benchmarks I have seen recently, e.g. [1][2]. The thing that I like with Armadillo is that you can just pick the BLAS implementation that is the fastest on your platform and run with it, while still being high-level compared to BLAS. [1] http://gcdart.blogspot.co.uk/2013/06/fast-matrix-multiply-and-ml.html http://gcdart.blogspot.co.uk/2013/06/fast-matrix-multiply-an... [2] https://github.com/tmolteno/necpp/issues/18 https://github.com/tmolteno/necpp/issues/18
  - srean 11y ago
    > you can just pick the BLAS implementation that is the fastest on your platform and run with it Just in case this is useful info, Eigen and Blaze works the same way, Blitz++ doesn't. Blaze seems to be doing the best in the benchmarks. > OpenBLAS is pretty much on par with Intel MKL I wont be surprised. I have seen ATLAS beat MKL on some BLAS functions on my runs. Sparse matrix multiply is particularly badly implemented in MKL (atleast was). One could beat their sparse multiply by using multiple instance of their own sparse matrix vector multiplies. But the reason I mentioned ICC is that there is much more to SIMD than just linear algebraic operations. Non-linear operations that come up quite frequently in stats/ML are sin, cos, exp, log, tanh etc on vectors. GCC/G++ does a decent job now (the best part is that it emits info why it wasn't able to vectorize a particular loop. This lets you restructure the loops to ai the compiler), but ICC still rules.
    - 11y ago
cmarschner 11y ago
For machine learning / neural nets in C++ there is also Caffe (for ConvNets) and CNTK (for almost any type of network).
plafl 11y ago
Writing scientific code is hard both in C++ and python, but reading and manipulating data and visualizing it is way easier in python. It's also easier to install and use third party libraries.
tdees40 11y ago
It's unimaginable for me to do most of this stuff without a REPL. That's the show-stopper for me with C++.
- leni536 11y ago
  Maybe there will be a working REPL for C++ in the future: https://github.com/vgvassilev/cling https://github.com/vgvassilev/cling

by leni536 11y ago

rasbt 11y ago

I agree with you. However, note that many people who are using Python for writing scientific code make use of C/C++ in one way or the other (aside from NumPy, SciPy, and Theano). For example, many people write the "most intensive" computations down in C/C++/Cython if they call those functions frequently -- Python becomes a wrapper. One example that pops into my mind is khmer (https://github.com/dib-lab/khmer https://github.com/dib-lab/khmer)

RA_Fisher 11y ago
I believe S (ancestor of R) started as C glue at Bell Labs. I've heard it said a few times that R is slow, but that doesn't really make much sense if your bottleneck routines are R calls to C++.
srean 11y ago
This is a common refrain: drop down to C, C++, Fortran for the computation intensive parts. It works, but only to a degree. The inefficiencies lie in the vectorization semantics of the host language(s) that leads to extra copies and extra levels of indirection. So this dual language mode of operation typically does not approach what one could have obtained had one disposed the baggage entirely, except for I/O. Usually in the quest for better speed, that is what remains, as one moves progressively larger portions of the application in the C, C++, Fortran part of the code. A reason I like Julia is that I can largely avoid this dual language annoyance, and enjoy the succinctness of pithy vectorized expressions using https://github.com/lindahua/Devectorize.jl https://github.com/lindahua/Devectorize.jl
- syllogism 11y ago
  You don't have to use the vectorization though. I prefer to work with Cython and just use pointers. You don't get to write vector + vector --- you have to actually write the loop --- but that's not such a big deal imo.
  - srean 11y ago
    I am lazy and super prone to make one-off errors, so that to me is the _biggest_deal_ TM :)
- deleted 11y ago
  [deleted]
- Lofkin 11y ago
  Numba for python also has a no copies vectorization mode.
- nxb 11y ago
  The speed performance of C/C++ is terrible, compared to GPU speed, which is often 50x to 100x faster. The goal has shifted and continues to shift in that direction. C/C++ is no longer best for speed, not even close. Now, all that matters is which language has the libraries that make it easiest to get custom code onto the GPU. Python and Lua seem to be winning there, by far.
  - leni536 11y ago
    >Now, all that matters is which library makes it easiest to get custom code onto the GPU. Python and Lua seem to be winning there, by far. This is interesting. How is it possible that python and lua have more efficient wrappers around GPU libraries? Also there are many GPU libraries for C/C++ too. Armadillo can use NVBLAS as a backend too. I'm not sure if I get your point of C/C++ being slow.
    - nxb 11y ago
      It's not about wrapping. The real power is in the cross-compilation of expressions and entire complex data pipelines, from a simple-as-possible high-level language into GPU language. That's the power at the core of, e.g. Theano.

cinquemb 11y ago

I see this a lot in my lab (days/weeks to run computations and analysis that could take minutes/hours), and I used to offer to help the postdocs/phd students port their code from matlab to C/C++, but I've mostly given up on offering unless they ask for help, or if it seems like fun. I also think it's because they mostly think they won't work on these things again (which is kind of weird to think about since they spent most of their life working to get to this point, but that's another conversation). Aside, I think armadillo is pretty great, especially going back and forth with other matrix libraries, also nice wrapper around OpenBLAS which made not running something on a cluster more bearable.

srean 11y ago

I saw you mention Armadillo a few times, seems you are having a lot of fun with it. Another somewhat look-alike is Eigen, its pretty nice too. These are all modern takes on the original sin Blitz++ which is actually more full featured in what they can represent than Eigen, Armadillo, mublas (by Boost) and Blaze: it supports multidimensional arrays as opposed to just (2d) matrices. Where Blitz++ falls short of the competition is full use of SIMD instructions. However, G++ does a plenty good job of vectorization, but the king of the hill is still Intel's compiler.

danieldk 11y ago
I am not sure about Intel's compiler, but OpenBLAS is pretty much on par with Intel MKL in most benchmarks I have seen recently, e.g. [1][2]. The thing that I like with Armadillo is that you can just pick the BLAS implementation that is the fastest on your platform and run with it, while still being high-level compared to BLAS. [1] http://gcdart.blogspot.co.uk/2013/06/fast-matrix-multiply-and-ml.html http://gcdart.blogspot.co.uk/2013/06/fast-matrix-multiply-an... [2] https://github.com/tmolteno/necpp/issues/18 https://github.com/tmolteno/necpp/issues/18
- srean 11y ago
  > you can just pick the BLAS implementation that is the fastest on your platform and run with it Just in case this is useful info, Eigen and Blaze works the same way, Blitz++ doesn't. Blaze seems to be doing the best in the benchmarks. > OpenBLAS is pretty much on par with Intel MKL I wont be surprised. I have seen ATLAS beat MKL on some BLAS functions on my runs. Sparse matrix multiply is particularly badly implemented in MKL (atleast was). One could beat their sparse multiply by using multiple instance of their own sparse matrix vector multiplies. But the reason I mentioned ICC is that there is much more to SIMD than just linear algebraic operations. Non-linear operations that come up quite frequently in stats/ML are sin, cos, exp, log, tanh etc on vectors. GCC/G++ does a decent job now (the best part is that it emits info why it wasn't able to vectorize a particular loop. This lets you restructure the loops to ai the compiler), but ICC still rules.
  - 11y ago

cmarschner 11y ago

For machine learning / neural nets in C++ there is also Caffe (for ConvNets) and CNTK (for almost any type of network).

plafl 11y ago

Writing scientific code is hard both in C++ and python, but reading and manipulating data and visualizing it is way easier in python. It's also easier to install and use third party libraries.

tdees40 11y ago

It's unimaginable for me to do most of this stuff without a REPL. That's the show-stopper for me with C++.

leni536 11y ago
Maybe there will be a working REPL for C++ in the future: https://github.com/vgvassilev/cling https://github.com/vgvassilev/cling