3 ms·
But people in the fields of computer vision, artificial intelligence, machine learning, concurrent data-structures and others – very smart, very forward thinkin
by michaelochurch 13y ago
But people in the fields of computer vision, artificial intelligence, machine learning, concurrent data-structures and others – very smart, very forward thinking people – would pick Java or C/C++ nine times out of ten, and not only because these languages are better suited to their task performance-wise, but because they have other, more important things to think about.
Yes and no. There are a few things to keep in mind.
First, these fields have a lot of finesse for which there's no substitute for years of trial-and-error. What's the right learning rate for a neural network? (There isn't one; real NN implementations have varying strategies for adapting the LR, but even picking the right one of those requires knowledge both of how NNs work, and the data itself, since noisier data means there won't be monotonic learning progress unless one trains over the whole set-- and that's usually impractical.) What this means is that programmers tend to have a longer lifespan and you don't see as much tech churn.
Also, most machine learning programmers use different languages for exploration and production. For the former, Python, R and Matlab seem to be the favorites. For the latter, C and Java are the traditional incumbents, though I could see Scala or Clojure breaking in. Making Clojure performant isn't elegant, but (a) it's not worse than writing Java or C++, and (b) the same is true of Common Lisp. Typically, though, production machine learning doesn't (or shouldn't) have a high LOC-count, especially because most ML projects are there to do things that would sprawl into unmaintainable complexity if done by hand (e.g. speech recognition).
Clojure won't replace C for low-level system programming that requires manual memory management, but even in machine learning, that doesn't apply to most of the work-- remember: half your time will be spend looking for patterns or exploring methods, many of which won't work-- unless you're working at a very large scale. For most of ML, the performance-intensive part is linear algebra, and you usually will end up using wrappers around Lapack (which is in Fortran) for that.
What I would say about the higher end of programming is that engineers care more about "the right tool for the job" and less about "the tool I like best". They're less wedded to one technology stack, so they're willing to use Java if it's the right language for their problem. They're definitely not your typical Java jockeys, though.
- tel 13y agoML is a great example since, as you mention your bottlenecks are likely solved by hanging your hat on Lapack and getting out. The complexity of the application derives from data manipulation and feature selection. The first is a definite win for higher-level languages and the second... might be, but is really too large a hodgepodge of techniques to say. I'm personally beginning to find that Haskell is really nice for this situation because it helps me separate stupid-programming-bugs from improper-algorithm-implementation-bugs very efficiently. This saves a lot of time when I'm looking at the output of a high-dimensional, non-deterministic program and wonder what might have gone wrong. And I've written some Clojure ML code that was quite close to Java speeds several years ago. For ML, you're either going to get the answer immediately or be waiting a while. As long as your constants for language choice aren't 2x-5x it's not a big deal. When everything is Fortran anyway, the constants aren't 2x-5x.
- pron 13y agoMachine learning is also, by a big margin, the most common of the areas I've mentioned and is therefore best positioned to benefit from "mass production" efficiency tools. It is moving from the master shop to the factory floor.
- tel 13y agoI kind of disagree with this. I think there are huge needs for "better language" kinds of improvements with ML that gives insight into how various pieces in a pipeline get combined, how tuning happens, how feature generation relates to input data. I don't consider ML to be terrifically repeatable today due to a lot of non-standard low-level tweaking that goes on at the top levels of work there. I think it's definitely moving, but I don't think it's nearly as mechanized as, say, AES is, given that the tradeoffs one needs to balance are significantly more complex. You can't just Machine Learn something as much as people may try.
- samatman 13y agoJust a tip - LAPACK and BLAS are available for OpenCL now. No need to link against Fortran, and you get to use the GPU. http://developer.amd.com/tools-and-sdks/cpu-development/amd-core-math-library-acml/ http://developer.amd.com/tools-and-sdks/cpu-development/amd-...
- michaelochurch 13y agoThat's a great thing to know-- and I wish I could give it 20 upvotes.