8 ms·
Java and Scala? Who uses that in ML? Python has long been the best language for ML, with some competition from Matlab.
by p1esk 11y ago
Java and Scala? Who uses that in ML? Python has long been the best language for ML, with some competition from Matlab.
- hellofunk 11y agoThat's a highly opinionated statement; lots of languages are used for machine learning, Python certainly has no monopoly on the field. There are significant entities using Clojure, Lua and other languages to do interesting things in this arena.
- chronic41 11y agoPython has a monopoly in the field. Just because 1% of users write in non Python languages means that machine learning is extremely diverse.
- RA_Fisher 11y agoThe cool thing is that you can use many languages to do ML! Once you know the concepts, they can be implemented in dynamically-typed languages like Python, R, or JavaScript, or statically-typed languages like Scala, Java or C++. Functional or imperative, there are many ways to skin the ML cat. We're lucky to have such a diverse set of tools!
- aiahopeful 11y agoScala is actually getting quite popular for machine learning, as it translates well to parallelization.
- jeffjose 11y agoWhat do you recommend is a good primer (something similar to the article OP posted) for Python folks?
- deleted 11y ago[deleted]
- vonnik 11y agoIt seems like a no-brainer that one of the largest programming languages in the world should have strong machine learning tools. Maybe the reason ML isn't done in Java and Scala is simply because the tools haven't been there. Python's great, but if you want to do ML/DL with distributed systems working with Hadoop and Spark, it makes more sense to do that in Java and Scala.
- minimaxir 11y agoSpark, and its machine-learning module MLlib, are Java/Scala, although it has a Python API.
- byefruit 11y agoPeople who do ML for a job? There's plenty of heavy duty machine learning libraries and implementations for big data platforms. Just spark alone has a fairly high quality one: http://spark.apache.org/docs/latest/mllib-guide.html http://spark.apache.org/docs/latest/mllib-guide.html
- RyanZAG 11y agoPython isn't used for much ML in the field from my experience. It is heavily used for teaching and learning about ML - but for actual production ML, I've seen mostly compiled languages. The main reason is that ML is highly parallelizable and Python isn't terribly good at that. Plus you need to crunch large datasets and speed becomes important. So, respectfully, lots of people use languages other than Python for ML, and I doubt if Python is even the largest deploy base of ML.
- DavidSJ 11y agoIn research, Python is extremely popular since all the number crunching is done with numpy or Theano, which use BLAS and CUDA.
- nostrademons 11y agoI've seen Python (and R) used all the time for exploratory ML. Do all of your feature extraction, feature selection, parameter tweaking, and backtesting in Python, and then once you have a model that works reasonably well, port the feature extraction for only the features that actually work well over to a compiled language like Java or C++, train your models on lots of data, and do your actual classification in the compiled language. Most ML is an iterative process, and the final model that's used in production is just the tip of the iceberg of the development work that went on. Python works as well for exploratory programming there as it does for any other domain.
- seanmcdirmid 11y agoI've seen Python used heavily in ML where much of the work is done on the GPU anyways. There are plenty of libraries for that, and Python's slow nature isn't as relevant (though it still becomes relevant when CPU processing is the bottleneck!).
- vosper 11y agoI work with a team that is doing large scale ML and CV on millions of pages and videos. It's a production system tied to millions of dollars in revenue. Everything's done in Python.
- deleted 11y ago[deleted]
- mindcrime 11y agoJava and Scala? Who uses that in ML? Python has long been the best language for ML You're kidding, right? Java has been extremely popular for ML for a long time. Not to cast any shade on Python, but I'd say Java and Python are roughly equivalent in this regard. Both have good libraries for various ML tasks and both are very popular in the domain. For reference, a quick search on mloss.org finds 84 projects identified as "Java" and 105 identified as "Python". So while Python has a small edge in sheer numbers I think that supports my assertion that they roughly equivalent in terms of their popularity for ML.
- mathgeek 11y agoI feel that much like programming in the corporate world, Python is often used to teach ML while Java is more often used to implement it.
- p1esk 11y agoTwo out of three most popular deep learning libraries have python front end (theano and caffe). The third one (torch) uses Lua.
- hellofunk 11y agoThere is a large amount of research, including some hedge fund models, that are exclusively using Clojure, on the JVM, for machine learning. Just because public libraries might often be in Python doesn't mean it is the language of choice for the big guys (it often isn't). You'd be surprised how much in-house ML stuff is done on the JVM.
- p1esk 11y agoI guess I should have clarified that by ML I really meant DL, as this is the most important area of ML currently. Is there anyone writing large CNNs or RNNs using Java or Scala or Clojure? Are there any widely used DL libraries based on those languages?
- pincubator 11y agoI favor Python as well, but from what I see in industry, Python is best for exploration/initial experiments and Java (and recently, Scala) is for production.
- chmullig 11y agoI feel like I've read literally this exact subthread on HN twice before. Am I going insane?
- solomatov 11y agoJava and Scala are the main languages of Spark, one of the most popular large scale machine learning tools out there.
- eranation 11y agoOur data scientists are learning Scala and Spark (MLLib) as a replacement for Python and R. So sure, maybe Python has long been the "best language for ML" but also one time in the not so far past "MySpace was the best social network"