5 ms·

This looks awesome. I've been itching to try out some ideas I have after having gone through Bishop's book, but I've been hesitant to write the algorithms from

This looks awesome. I've been itching to try out some ideas I have after having gone through Bishop's book, but I've been hesitant to write the algorithms from scratch. Now I'll have to decide between learning matlab or a library such as this.

c0g 14y ago
Speaking as a PhD student in machine learning- Implement the algorithm yourself, first, in Python+Numpy. The only reason I feel comfortable with Gaussian Processes and SVMs is due to writing code to solve them manually. Once you're happy with the basics, and can test your ideas with code you intimately understand, optimise for speed by using a library like this.
- misiti3780 14y ago
  Implementing the SVM from scratch was time consuming - no?
  - iskander 14y ago
    The only tricky part would be writing a quadratic solver. Alternatives: either solve a linear SVM using gradient descent (simpler to write), or offload the core of the algorithm to an existing solver like cvxopt. edit: For an example of using cvxopt, check out http://www.mblondel.org/journal/2010/09/19/support-vector-machines-in-python/ http://www.mblondel.org/journal/2010/09/19/support-vector-ma...
    - droz 14y ago
      Another approach is to implement Platt's SMO: http://en.wikipedia.org/wiki/Sequential_minimal_optimization http://en.wikipedia.org/wiki/Sequential_minimal_optimization
    - misiti3780 14y ago
      cool - thanks
  - c0g 14y ago
    Yes it was :) but time well spent imo. On reflection I guess I might have had more free time to spend on this than a normal person - I did the SVM as a [small] part of my masters project, so if you're time constrained with a real job and a life then might be best to disregard me.
    - misiti3780 14y ago
      If you had the quadratic solver, I would think it would be reasonable to add the rest of the code. If you started adding costs, gammas, etc. I would think it would take a while. I spent hours looking at the source code of libSVM at my last job and never really understood what the hell was going on
- misiti3780 14y ago
  I do agree with you regardless, just curious
tlarkworthy 14y ago
You should definitely use a scriptable ML library. The process is very iterative and not suited to a compiled language like C++. I use skilearn alot, but also the matlab toolboxes or R are great. At its heart ML is alot of stats, so use something built for maths, not C++. It doesn't really make sense to break out C++ until you know exactly what algorithm and settings you need and your application is real time.
- bch 14y ago
  If the OP was thinking of writing his own algorithms, and this is a linkable library with that heavy math already implemented, couldn't he write bindings for Python/Lua/Tcl/Ruby and have everything he needs for script-ability, or am I missing something?
  - andrewguenther 14y ago
    You aren't missing anything, you are absolutely correct, but the question isn't "can they?" the question is "will they?"
    - bch 14y ago
      I'm in the Stanford/Coursera machine learning course right now, and something like this is nearly excactly what I've been looking for. As some others have said, GPLv3 is off-putting, but there is the LGPL mlpack lib (http://www.mlpack.org/ http://www.mlpack.org/) (also C++). Personally, project-wise, the only way this could be improved is if the project were pure C, and a BSD, MIT, or similar license. Quite looking forward to checking these out, though.
      - tlarkworthy 14y ago
        honestly you guys are crazy if you think shark is gonna help you learn machine learning. Its ideal for deployment of ML on things like embedded computer, robotics, games etc. where real time learning is required. Machine learning requires alot of experimentation and C++ is a terrible medium for that. There are loads of good machine learning libraries implemented for python and matlab. Pretty much every good paper in machine learning is accompanied by an algorithm implemented in matlab or python or R. learn using those reference designs. Once you figured out what you want, then deploy on a system in C++ by all means using shark. I do robotics for a living, and I do go from scripting to C++. Unless its absolutely necessary I avoid C++. Only things like vision which is so CPU hungry that it has its own computer do I require C++, every other algorithm stays in python.

c0g 14y ago

Speaking as a PhD student in machine learning- Implement the algorithm yourself, first, in Python+Numpy. The only reason I feel comfortable with Gaussian Processes and SVMs is due to writing code to solve them manually. Once you're happy with the basics, and can test your ideas with code you intimately understand, optimise for speed by using a library like this.

misiti3780 14y ago
Implementing the SVM from scratch was time consuming - no?
- iskander 14y ago
  The only tricky part would be writing a quadratic solver. Alternatives: either solve a linear SVM using gradient descent (simpler to write), or offload the core of the algorithm to an existing solver like cvxopt. edit: For an example of using cvxopt, check out http://www.mblondel.org/journal/2010/09/19/support-vector-machines-in-python/ http://www.mblondel.org/journal/2010/09/19/support-vector-ma...
  - droz 14y ago
    Another approach is to implement Platt's SMO: http://en.wikipedia.org/wiki/Sequential_minimal_optimization http://en.wikipedia.org/wiki/Sequential_minimal_optimization
  - misiti3780 14y ago
    cool - thanks
- c0g 14y ago
  Yes it was :) but time well spent imo. On reflection I guess I might have had more free time to spend on this than a normal person - I did the SVM as a [small] part of my masters project, so if you're time constrained with a real job and a life then might be best to disregard me.
  - misiti3780 14y ago
    If you had the quadratic solver, I would think it would be reasonable to add the rest of the code. If you started adding costs, gammas, etc. I would think it would take a while. I spent hours looking at the source code of libSVM at my last job and never really understood what the hell was going on
misiti3780 14y ago
I do agree with you regardless, just curious

tlarkworthy 14y ago

You should definitely use a scriptable ML library. The process is very iterative and not suited to a compiled language like C++. I use skilearn alot, but also the matlab toolboxes or R are great. At its heart ML is alot of stats, so use something built for maths, not C++. It doesn't really make sense to break out C++ until you know exactly what algorithm and settings you need and your application is real time.

bch 14y ago
If the OP was thinking of writing his own algorithms, and this is a linkable library with that heavy math already implemented, couldn't he write bindings for Python/Lua/Tcl/Ruby and have everything he needs for script-ability, or am I missing something?
- andrewguenther 14y ago
  You aren't missing anything, you are absolutely correct, but the question isn't "can they?" the question is "will they?"
  - bch 14y ago
    I'm in the Stanford/Coursera machine learning course right now, and something like this is nearly excactly what I've been looking for. As some others have said, GPLv3 is off-putting, but there is the LGPL mlpack lib (http://www.mlpack.org/ http://www.mlpack.org/) (also C++). Personally, project-wise, the only way this could be improved is if the project were pure C, and a BSD, MIT, or similar license. Quite looking forward to checking these out, though.
    - tlarkworthy 14y ago
      honestly you guys are crazy if you think shark is gonna help you learn machine learning. Its ideal for deployment of ML on things like embedded computer, robotics, games etc. where real time learning is required. Machine learning requires alot of experimentation and C++ is a terrible medium for that. There are loads of good machine learning libraries implemented for python and matlab. Pretty much every good paper in machine learning is accompanied by an algorithm implemented in matlab or python or R. learn using those reference designs. Once you figured out what you want, then deploy on a system in C++ by all means using shark. I do robotics for a living, and I do go from scripting to C++. Unless its absolutely necessary I avoid C++. Only things like vision which is so CPU hungry that it has its own computer do I require C++, every other algorithm stays in python.