4 ms·

Very interesting, but as a daily practitioner I am skeptical. First, this is a lot of code! As a C++ machine learning programmer, I am impressed as I know the

• pmelendez 13y ago
  May I ask how do you become a C++ ML programmer? It is not an usual position but one I would definitely be interested in pursuing
  • pilooch 13y ago
    Sure, I started as a researcher 14 years ago, then drifted to what I thought was a sweet spot then, half-research / half-programmer. I say 'sweet spot' because many applications did require both the academic and the applied background at the time, so for the sake of thrilling applications, it was worth 'downgrading' to pure engineering work when needed. Now I believe the game has changed a bit, coursera and others are infusing the minds of engineers with highly technical knowledge far more rapidly than before. Typically I am astonished at the number of implementations of deep learning techniques (Shark does include some, AFAIK). My past experience is that I had write many AI algorithms myself because I could not find any suitable, free and/or open implementations (or other researchers would not share theirs ;) ).
    • pmelendez 13y ago
      Thanks! That's very inspiring :) I worked in a similar position for a short time long time ago but after that I hadn't been able to find something similar. Good thing to see that people can find those sweet spots! :D It's very true what you said about all the implementations available now. Although, for most algorithms I tend to try to implement them myself as a learning experience, maybe the biggest exception is standard SVM since it is kinda tricky but even for that there are some online algorithms that are easy to implement. Thanks for sharing your experience!
• Schneekatze 13y ago
  Hi, shark developer here. First of all, we are glad that our library is discussed on this board! We are happy for every feedback we can get! Regarding Performance: we try to get the key algorithms as fast as possible. And for the hardest parts we rely not on ublas, but use optional bindings to ATLAS. Speed was one of the key design criteria. We hope that we achieved that. Clearly this is no guarantee that every algorithm is fast, but in this case: just add a ticket! Please bear in mind, that Shark is still in beta stage, and we are heavily developing it (I am right now working on the family of multi class SVMs). So for example parallelism using OpenMP is not fully integrated.
  • jhartmann 13y ago
    You should consider using Eigen for linear algebra, I have personally found it much better performance wise than using bindings to ATLAS or other more standard linear algebra solutions. ML algorithms tend to be multistage (think about the update of weights with momentum in a Neural network for example), and the primitives available in ATLAS or a blas library are really too low level. Eigen since it generates code for a whole complicated expression can blow a standard linear algebra library out of the water for a certain class of problems. For others obviously the highly tuned vendor BLAS code would win, but I've seen huge speedups by using Eigen it fits well for complex ML operations.
    • Schneekatze 13y ago
      This is true, even though we would rather switch to Armadillo due to it's easier handling and better high-level behaviour. Right now the linear algebra library we use -ublas- has the same behaviour as Eigen for BLAS1 type expressions. So it tries to generate optimal (non-SSE) code. Only for BLAS2 and 3 we fall back to the ATLAS-routines which has the same performance as Eigen on the interesting problem sizes. //small edit In the end it is not so interesting whether the BLAS1-type expressions are fast as they make up < 1% of run time performance. The big chunks are the data processing inside the matrix-matrix multiplications of the Neural Networks and similar entities.
      • 13y ago