5 ms·

Machine Learning in JavaScript

169 points by xd 13y ago

Already__Taken 13y ago
This blog also contains a JavaScript physics series which is very cool. Do hope this author writes more again it has been quite.
- bkanber 13y ago
  Thanks! -- I am intending to write more, but the pace is sloooowww.
Joe8Bit 13y ago
If you're looking for more on this, or some general purpose JS NLP you'd do worse than to checkout node-natural[0] [0]: https://github.com/NaturalNode/natural https://github.com/NaturalNode/natural
frik 13y ago
That's great, keep up the good work. For some reason it's somewhat hard to find C-style science code examples in some disciplines. Python feels a bit like a plague in this respect. Everytime I have to wrap my head around while converting code to C-like language (C, C++, PHP, JS).
- tlarkworthy 13y ago
  The distance to convert math to python is so much shorter than math to C or math to javascript. You need something like numpy to make working in javasctipt easier before there will be a proliferation of of ML in JS. I really love JS for its distribution and some of the visualizations are amazing. But the low level, numerically stable, matrix math primitives are sorely lacking.
  - Already__Taken 13y ago
    I feel the people doing heavy work with graphics APIs and 3D work with robots too will push this back into the javascript language given some time.
    - tlarkworthy 13y ago
      I work with robots. The leading middleware is ROS. It is multi language, but does not support JS out the box (unlike LISP, Java, C++ and Python), though there is movement there http://brandonalexander.com/rosnodejs/ http://brandonalexander.com/rosnodejs/ I can see JS useful for a UI to a robot, but I can't see it replacing Python for math, or C++ for speed, or LISP for planning systems. That said, I can imagine node.js being a better async message router than the current C++ one. ROS is glued together with XML-RPC, which I think was a mistake (why not JSON???)
  - nashequilibrium 13y ago
    You need the love of the data science community and they have settled on python, and when they really want to scale they use Java. I noticed Julia becoming very popular and the twitter guys use scala for their stuff.
viana007 13y ago
For neural networks, you can use BrainJS https://github.com/harthur/brain https://github.com/harthur/brain
- nightski 13y ago
  So the input/output pairs are a linked list of objects? Which then contain vectors comprised of linked lists? I am not very into JavaScript, but that right there must preclude this from doing anything significant in a reasonable amount of time?
LambdaAlmighty 13y ago
Only a question of time before the Machine Learning wave arrived at the arguably least suited--but certainly the most popular--platform! There's money to be made with this combination. The field is ripe. Good write up too.
code_scrapping 13y ago
I still view JS as a UI-oriented language, and I really don't know why would you want to implement processor-heavy algorithms in a browser environment, which need a lot of data and don't use the networking. I would still stick to python. Or java. Or anything else which has a clear syntax and can run at a useful speed (I'm not mentioning C++ because of the coding overhead and dirty tricks which makes it a bit unfriendly for learning an algorithm)
- xd 13y ago
  "Through this series of articles, I’ll teach you the fundamental machine learning algorithms using Javascript"
- dangoor 13y ago
  Clarity of syntax is a matter of opinion (personally, I agree that Python is clearer than JS... Java, not so much.) Implying that JavaScript can't "run at a useful speed" is wrong, using modern implementations. This is especially true for code that runs through lots of repetition as the just-in-time compilers in the JS engines do a remarkable job. Not to mention that viewing JS as a UI-oriented language seems a bit out of date given the 40k or so packages for Node.js that are in npm. JavaScript of today is pretty different than JS of 2007, and there are more changes coming with generators, iterators, destructuring, class syntax, arrow functions, promises, etc.
  - Joe8Bit 13y ago
    While I disagree with the comment you're responding to, and agree with yours, there are some interesting problems doing resource heavy operations in ML/NLP in an environment like Node that's inherently single threaded. I'm actually adding multi-threading to classifier training in node-natural as we speak [0] so it's something I'm recently familiar with. Multi-threading in JS isn't new or particularly exciting (even less so is multithreading in ML/NLP applications) but the marriage of the two has led to a few interesting problems in JS's asynchronous/event based view of the world! [0]: https://github.com/NaturalNode/natural/issues/124 https://github.com/NaturalNode/natural/issues/124 -- Edited for clarity
bkanber 13y ago
Author here -- thanks for submitting, xd! Let me know if you have any questions. I do intend to keep up with this series, although my pace is pretty slow at about one article every three months or so. There are already a couple of comments about running ML in JS and how JS and the browser environment isn't terribly suited for heavy calculations. First: you're totally correct; second, I chose JS because it's 1) accessible -- whether you're Python or Ruby or PHP on the backend, you're probably comfortable with JS and 2) it demystifies machine learning -- you have to write your ML from scratch, without the help of all those wonderful Python libs, and I think this exercise shows you that it's not so mysterious after all. Anyway, thanks for reading, and I'll poke in here throughout the day if you have questions.
- xd 13y ago
  No no, thank you! I've been building a data management platform for the last 8 years and we are now at the stage where we want to provide tools to help our customers get more from their data than just statistics. As my programming experience is mainly in PHP and JS this set of articles is helping me grasp ML rather than trying to wrap my head around a new language. I'm currently working on k-means clustering and re-implementing everything in PHP to get the best possible understanding I can .. my aim after that is to see how well I can implement things at an SQL level.
  - bkanber 13y ago
    Excellent, I'm glad to hear it! If you ever want to reach out, feel free -- email in my profile.
    - ttrobertson 13y ago
      How did you first come to get interested in JavaScript?
- d_j_b 13y ago
  This is great stuff - thank you very much. >all those wonderful Python libs As a non-mathematician I have no understanding of how wonderful they really are, which is why this sort of thing is so valuable.
TeeWEE 13y ago
X in JavaScript..... ugh if all you have is a hammer, everything looks like a nail
- bkanber 13y ago
  Have you read the article? I make it pretty apparent that JS is used primarily for its educational value :)
  - TeeWEE 13y ago
    To be honest i didn't initially. I just read it. I think it is a noble thing to explain this in JS. But i don't think "because every body uses js" is a good reason to choose js. However your specific use case makes sense. But in a broader sense I see more and more people fleeing to JS because its what they know.
    - thrush 13y ago
      What alternatives would you recommend for someone new to programming and CS? I guess fleeing implies that they were using other tools already, but a lot of new devs are going to JS because it just makes sense to start there (lots of flexibility, hyperactive community, education value).
      - stusmall 13y ago
        If you are at the point of your CS education that you are taking a serious look at machine learning and understanding the theory then you shouldn't have a problem translating into whatever your language of choice is. I get why a teacher would just want to pick a language and say "this is what it is in" but I don't get people who need CS concepts taught in their language of choice. The hard part is the theory and not the implementation. When I took it in university it was taught in language agnostic psuedocode and we were free to use any language from a long list for our assignments.
        13y ago
morganherlocker 13y ago
To those wondering why someone would want ML in JS, there are loads of reasons. For starters, node.js, which makes most of the arguments regarding server/client moot. Secondly, there are many client side applications for these types of algorithms as well. K-means clustering, for example, is already used by many mapping libraries to group together large numbers of points[1]. I personally use neural networks and affinity propagation in many of my applications for predictive analysis. This does not have to only be educational, or of a 'toy' nature. [1] http://danzel.github.io/Leaflet.markercluster/example/marker-clustering-realworld.388.html http://danzel.github.io/Leaflet.markercluster/example/marker...
- nashequilibrium 13y ago
  Nodejs is for i/o, i know it has workarounds for long running tasks(threadpool) but it does not excell at that. Training your model, updating your model, validation, matrix factorization on large datasets etc, i just don't see how Javascript helps here. Maybe just taking the http request and dumping it onto a rabbitmq queue to to classify something but you still have a whole host of other stuff to deal with.
  - thrush 13y ago
    I think they are making the argument for capability rather than best of class. This is really exciting for Javascript and NodeJS devs right now because of the "Give me an inch and I'll take a foot" mentality.
  - morganherlocker 13y ago
    > Nodejs is for i/o Node is a general purpose language that can be used for all kinds of things. I switched from Python to node.js about a year ago for exactly the sort of tasks you are describing and could not be happier. Right off the bat I had huge speed improvements. Also, io is one of the biggest issues with web based data analysis, so it really should not be underestimated. I can do more with less with node than I could with Python. This is especially true with long running tasks where a 1 minute processing time vs a 20 minute processing time might mean you need 1/20th the number of servers in a cluster ($$$). Of course, this could be a pretty good argument for something even faster/lower level, but for me, node.js struck a good balance between performance and ease of development/ecosystem. As usual, YMMV. One last point. The language you choose cannot always be the best language for every task you need. Typically you choose a stack based on the most common/important tasks in your infrastructure, then for less common tasks you just make it work with what the chosen language provides. In this case node.js does not need to be the best solution for ML, it just needs to check the box for being possible, so that devs who needed node.js for other reasons now have the ability to add ML to their toolbox.
ecesena 13y ago
Cool project! Have you already tried asm.js and/or measured performances?
- frik 13y ago
  You cannot write asm.js by hand (in a sane way... it uses one big array for everything). It's meant to be translated from emscripten clang compiler project. So you can compile C/C++ code to asm.js. But Javascript engines like V8 with its JIT are way faster than Python. You can even use typed arrays that give you almost native speed for such operations (e.g. matrix). I am coding a 3D game in WebGL and JS is as fast as Java when used in a modern fashion, though JS run in every browser
kajecounterhack 13y ago
For those who aren't familiar Andrej Karpathy has done a lot of cool stuff with ML in JS. Particularly he has a CNN library -- deep learning comes to JS! http://cs.stanford.edu/people/karpathy/convnetjs/ http://cs.stanford.edu/people/karpathy/convnetjs/ http://cs.stanford.edu/people/karpathy/svmjs/demo/ http://cs.stanford.edu/people/karpathy/svmjs/demo/ Heather Arthur (npm libraries brain, classifier) has also done a bunch of cool stuff! https://github.com/harthur https://github.com/harthur
e12e 13y ago
Interesting, but I wonder about this: > … well, most of the time. There are some things you really can’t do in PHP or Javascript, but those are the more advanced algorithms that require heavy matrix math. Leaving out javascript (in the browser), it sounds like an odd statement to make about php -- after all one of php strengths is how easy it is to link with c-libraries (or other with c ffi)? Among other things I quickly found: http://www.php.net/manual/en/intro.lapack.php http://www.php.net/manual/en/intro.lapack.php
perimo 13y ago
I kicked around with some JS manifold learning stuff[1] a while back for essentially the same purpose: practice in writing things from scratch, while making it easier for other people to play with. [1]: https://github.com/perimosocordiae/js_manifolds https://github.com/perimosocordiae/js_manifolds

169 points by xd 13y ago

This blog also contains a JavaScript physics series which is very cool. Do hope this author writes more again it has been quite.

bkanber 13y ago
Thanks! -- I am intending to write more, but the pace is sloooowww.

Joe8Bit 13y ago

If you're looking for more on this, or some general purpose JS NLP you'd do worse than to checkout node-natural[0] [0]: https://github.com/NaturalNode/natural https://github.com/NaturalNode/natural

frik 13y ago

That's great, keep up the good work. For some reason it's somewhat hard to find C-style science code examples in some disciplines. Python feels a bit like a plague in this respect. Everytime I have to wrap my head around while converting code to C-like language (C, C++, PHP, JS).

tlarkworthy 13y ago
The distance to convert math to python is so much shorter than math to C or math to javascript. You need something like numpy to make working in javasctipt easier before there will be a proliferation of of ML in JS. I really love JS for its distribution and some of the visualizations are amazing. But the low level, numerically stable, matrix math primitives are sorely lacking.
- Already__Taken 13y ago
  I feel the people doing heavy work with graphics APIs and 3D work with robots too will push this back into the javascript language given some time.
  - tlarkworthy 13y ago
    I work with robots. The leading middleware is ROS. It is multi language, but does not support JS out the box (unlike LISP, Java, C++ and Python), though there is movement there http://brandonalexander.com/rosnodejs/ http://brandonalexander.com/rosnodejs/ I can see JS useful for a UI to a robot, but I can't see it replacing Python for math, or C++ for speed, or LISP for planning systems. That said, I can imagine node.js being a better async message router than the current C++ one. ROS is glued together with XML-RPC, which I think was a mistake (why not JSON???)
- nashequilibrium 13y ago
  You need the love of the data science community and they have settled on python, and when they really want to scale they use Java. I noticed Julia becoming very popular and the twitter guys use scala for their stuff.

viana007 13y ago

For neural networks, you can use BrainJS https://github.com/harthur/brain https://github.com/harthur/brain

nightski 13y ago
So the input/output pairs are a linked list of objects? Which then contain vectors comprised of linked lists? I am not very into JavaScript, but that right there must preclude this from doing anything significant in a reasonable amount of time?

LambdaAlmighty 13y ago

Only a question of time before the Machine Learning wave arrived at the arguably least suited--but certainly the most popular--platform! There's money to be made with this combination. The field is ripe. Good write up too.

code_scrapping 13y ago

I still view JS as a UI-oriented language, and I really don't know why would you want to implement processor-heavy algorithms in a browser environment, which need a lot of data and don't use the networking. I would still stick to python. Or java. Or anything else which has a clear syntax and can run at a useful speed (I'm not mentioning C++ because of the coding overhead and dirty tricks which makes it a bit unfriendly for learning an algorithm)

xd 13y ago
"Through this series of articles, I’ll teach you the fundamental machine learning algorithms using Javascript"
dangoor 13y ago
Clarity of syntax is a matter of opinion (personally, I agree that Python is clearer than JS... Java, not so much.) Implying that JavaScript can't "run at a useful speed" is wrong, using modern implementations. This is especially true for code that runs through lots of repetition as the just-in-time compilers in the JS engines do a remarkable job. Not to mention that viewing JS as a UI-oriented language seems a bit out of date given the 40k or so packages for Node.js that are in npm. JavaScript of today is pretty different than JS of 2007, and there are more changes coming with generators, iterators, destructuring, class syntax, arrow functions, promises, etc.
- Joe8Bit 13y ago
  While I disagree with the comment you're responding to, and agree with yours, there are some interesting problems doing resource heavy operations in ML/NLP in an environment like Node that's inherently single threaded. I'm actually adding multi-threading to classifier training in node-natural as we speak [0] so it's something I'm recently familiar with. Multi-threading in JS isn't new or particularly exciting (even less so is multithreading in ML/NLP applications) but the marriage of the two has led to a few interesting problems in JS's asynchronous/event based view of the world! [0]: https://github.com/NaturalNode/natural/issues/124 https://github.com/NaturalNode/natural/issues/124 -- Edited for clarity

bkanber 13y ago

Author here -- thanks for submitting, xd! Let me know if you have any questions. I do intend to keep up with this series, although my pace is pretty slow at about one article every three months or so. There are already a couple of comments about running ML in JS and how JS and the browser environment isn't terribly suited for heavy calculations. First: you're totally correct; second, I chose JS because it's 1) accessible -- whether you're Python or Ruby or PHP on the backend, you're probably comfortable with JS and 2) it demystifies machine learning -- you have to write your ML from scratch, without the help of all those wonderful Python libs, and I think this exercise shows you that it's not so mysterious after all. Anyway, thanks for reading, and I'll poke in here throughout the day if you have questions.

xd 13y ago
No no, thank you! I've been building a data management platform for the last 8 years and we are now at the stage where we want to provide tools to help our customers get more from their data than just statistics. As my programming experience is mainly in PHP and JS this set of articles is helping me grasp ML rather than trying to wrap my head around a new language. I'm currently working on k-means clustering and re-implementing everything in PHP to get the best possible understanding I can .. my aim after that is to see how well I can implement things at an SQL level.
- bkanber 13y ago
  Excellent, I'm glad to hear it! If you ever want to reach out, feel free -- email in my profile.
  - ttrobertson 13y ago
    How did you first come to get interested in JavaScript?
d_j_b 13y ago
This is great stuff - thank you very much. >all those wonderful Python libs As a non-mathematician I have no understanding of how wonderful they really are, which is why this sort of thing is so valuable.

TeeWEE 13y ago

X in JavaScript..... ugh if all you have is a hammer, everything looks like a nail

bkanber 13y ago
Have you read the article? I make it pretty apparent that JS is used primarily for its educational value :)
- TeeWEE 13y ago
  To be honest i didn't initially. I just read it. I think it is a noble thing to explain this in JS. But i don't think "because every body uses js" is a good reason to choose js. However your specific use case makes sense. But in a broader sense I see more and more people fleeing to JS because its what they know.
  - thrush 13y ago
    What alternatives would you recommend for someone new to programming and CS? I guess fleeing implies that they were using other tools already, but a lot of new devs are going to JS because it just makes sense to start there (lots of flexibility, hyperactive community, education value).
    - stusmall 13y ago
      If you are at the point of your CS education that you are taking a serious look at machine learning and understanding the theory then you shouldn't have a problem translating into whatever your language of choice is. I get why a teacher would just want to pick a language and say "this is what it is in" but I don't get people who need CS concepts taught in their language of choice. The hard part is the theory and not the implementation. When I took it in university it was taught in language agnostic psuedocode and we were free to use any language from a long list for our assignments.
      - 13y ago

morganherlocker 13y ago

To those wondering why someone would want ML in JS, there are loads of reasons. For starters, node.js, which makes most of the arguments regarding server/client moot. Secondly, there are many client side applications for these types of algorithms as well. K-means clustering, for example, is already used by many mapping libraries to group together large numbers of points[1]. I personally use neural networks and affinity propagation in many of my applications for predictive analysis. This does not have to only be educational, or of a 'toy' nature. [1] http://danzel.github.io/Leaflet.markercluster/example/marker-clustering-realworld.388.html http://danzel.github.io/Leaflet.markercluster/example/marker...

nashequilibrium 13y ago
Nodejs is for i/o, i know it has workarounds for long running tasks(threadpool) but it does not excell at that. Training your model, updating your model, validation, matrix factorization on large datasets etc, i just don't see how Javascript helps here. Maybe just taking the http request and dumping it onto a rabbitmq queue to to classify something but you still have a whole host of other stuff to deal with.
- thrush 13y ago
  I think they are making the argument for capability rather than best of class. This is really exciting for Javascript and NodeJS devs right now because of the "Give me an inch and I'll take a foot" mentality.
- morganherlocker 13y ago
  > Nodejs is for i/o Node is a general purpose language that can be used for all kinds of things. I switched from Python to node.js about a year ago for exactly the sort of tasks you are describing and could not be happier. Right off the bat I had huge speed improvements. Also, io is one of the biggest issues with web based data analysis, so it really should not be underestimated. I can do more with less with node than I could with Python. This is especially true with long running tasks where a 1 minute processing time vs a 20 minute processing time might mean you need 1/20th the number of servers in a cluster ($$$). Of course, this could be a pretty good argument for something even faster/lower level, but for me, node.js struck a good balance between performance and ease of development/ecosystem. As usual, YMMV. One last point. The language you choose cannot always be the best language for every task you need. Typically you choose a stack based on the most common/important tasks in your infrastructure, then for less common tasks you just make it work with what the chosen language provides. In this case node.js does not need to be the best solution for ML, it just needs to check the box for being possible, so that devs who needed node.js for other reasons now have the ability to add ML to their toolbox.

ecesena 13y ago

Cool project! Have you already tried asm.js and/or measured performances?

frik 13y ago
You cannot write asm.js by hand (in a sane way... it uses one big array for everything). It's meant to be translated from emscripten clang compiler project. So you can compile C/C++ code to asm.js. But Javascript engines like V8 with its JIT are way faster than Python. You can even use typed arrays that give you almost native speed for such operations (e.g. matrix). I am coding a 3D game in WebGL and JS is as fast as Java when used in a modern fashion, though JS run in every browser

kajecounterhack 13y ago

For those who aren't familiar Andrej Karpathy has done a lot of cool stuff with ML in JS. Particularly he has a CNN library -- deep learning comes to JS! http://cs.stanford.edu/people/karpathy/convnetjs/ http://cs.stanford.edu/people/karpathy/convnetjs/ http://cs.stanford.edu/people/karpathy/svmjs/demo/ http://cs.stanford.edu/people/karpathy/svmjs/demo/ Heather Arthur (npm libraries brain, classifier) has also done a bunch of cool stuff! https://github.com/harthur https://github.com/harthur

e12e 13y ago

Interesting, but I wonder about this: > … well, most of the time. There are some things you really can’t do in PHP or Javascript, but those are the more advanced algorithms that require heavy matrix math. Leaving out javascript (in the browser), it sounds like an odd statement to make about php -- after all one of php strengths is how easy it is to link with c-libraries (or other with c ffi)? Among other things I quickly found: http://www.php.net/manual/en/intro.lapack.php http://www.php.net/manual/en/intro.lapack.php

perimo 13y ago

I kicked around with some JS manifold learning stuff[1] a while back for essentially the same purpose: practice in writing things from scratch, while making it easier for other people to play with. [1]: https://github.com/perimosocordiae/js_manifolds https://github.com/perimosocordiae/js_manifolds