11 ms·

I feel I'm in a somewhat unique position to talk about easy/hardness of machine learning; I've been working for several months on a project with a machine learn

• jasonwatkinspdx 13y ago
  Good point. However, the author does recommend using open source ML libraries, so applying the techniques may not require some of the implementation knowledge you suggest. It's likely more relevant to understand the semantic constraints and pitfalls (eg, am I over-fitting right now?) since the work of a small number of talented implementers can be easily reused.
• pmelendez 13y ago
  I was going to say something around those lines but you save me the effort. Although in my opinion using ML and have some decent results is not that hard. Use it and being close to the state in the art (i.e improving your recognition rate from 65% to 80%) is a completely different matter and it is hard and it requires a fully understanding of the details of the algorithm in use. Still it doesn't meant that you can't do interesting things with an overall understanding of machine learning, but making a good cat detector is hard [0] [1] [0] http://research.google.com/archive/unsupervised_icml2012.html http://research.google.com/archive/unsupervised_icml2012.htm... [1] http://techtalks.tv/talks/machine-learning-and-ai-via-brain-simulations/57862/ http://techtalks.tv/talks/machine-learning-and-ai-via-brain-...
  • dylandrop 13y ago
    For a 65% to 80% jump, I'd say it depends on the subfield. With NLP (which is basically applied machine learning), it's not uncommon to see algorithms with >80% accuracy.
• csmatt 13y ago
  Does there exist a tool that will translate mathematical symbols or an entire equation of them into plain English?
  • coolsunglasses 13y ago
    Mathematical symbols are overloaded by context, this could be done but you'd still need to tell it whether it was a formula from "machine learning" or "topology". Then you'd still fail to account for symbols invented/repurposed in a single paper or by a single author. Basically, the tool itself is a machine learning problem.
    • btown 13y ago
      Even the simple case of taking a PDF, finding the first use of any given symbol in that paper, and turning the PDF into an interactive document where hovering on any use of the symbol shows a tooltip showing the first use - now that would be both useful and feasible. But... the Venn diagram of people able to do such a thing, people motivated to do such a thing (i.e. people reading academic papers who aren't math whizzes), and people with the time to do such a task... seems not to have a sufficient intersection. Most of us in this thread fulfill categories 1 and 2, but not 3... come to think of it, I have work to do... ducks.
      • wetmore 13y ago
        Some friends and I are trying to do something similar actually.
  • chestervonwinch 13y ago
    Sounds like a problem for machine learning :)
  • jasonwatkinspdx 13y ago
    A better approach might be translating them into more uniform s-expressions. Mathematica does this. You can also check out a similar approach in Structure and Interpretation of Classical Mechanics where tensor expressions are reduced to scheme expressions.
• mistermcgruff 13y ago
  I agree that the notation is tough. Especially for folks who need to learn to do this stuff as a one-off and not as a career. That's why I wrote a book that teaches the algorithms with nearly no math notation while trying to be very rigorous: http://www.amazon.com/Data-Smart-Science-Transform-Information/dp/111866146X http://www.amazon.com/Data-Smart-Science-Transform-Informati...
• diab0lic 13y ago
  This response, and the replies to it actually come as a bit of a surprise to me. I am a graduate student in artificial intelligence who has a comparatively weak background in mathematics who is still very capable of applying, implementing, and improving upon machine learning techniques. I have never taken a calculus or linear algebra class. (Don't ask how I managed that, an effect of the odd route I took to get where I am). The point is you shouldn't be telling yourself you can't do it because you don't know the math, you may surprise yourself. That being said there are plenty of barriers to a software engineer picking up some papers on machine learning and getting started. These are the result of some serious issues that I believe exist with academia at present and specifically the "publication game". I will not however get into this as this comment will quickly turn into the longest paper I've ever written. It is a problem with the system, and as the reader it is not your fault. Your issue with applied machine learning may be largely imagined. Take for example the k-means clustering algorithm in the article. This is quite useful in a number of places be it data analysis or intelligent user interfaces, its straightforward to implement, and many usable implementations exist online. The same case is true for other models such as linear regression, naive bayes, even things like support vector machines are pretty easy to use these days. In any case, as selfish as this is, this sentiment you're expressing provides some comfort that I'll be in demand when I enter the job market. :P Though if you really want to implement and apply some simple yet useful algorithms he recommends Programming Collective Intelligence at the end of the article and I can definitely second that recommendation.
  • andreasvc 13y ago
    I think you're glossing over the fact that for a non-trivial problem, you have the pick the right machine learning algorithm from a large set of possibilies, and typically you have to set a number of parameters to make it work. On top of that you have to follow proper methodology to get meaningful results; e.g., to determine whether the accuracy you're getting generalizes to other data as well. I think all of this contributes to making applied machine learning a non-trivial skill. And indeed, AI is just the right background for that, but the point is that you need such a background.
    • 13y ago
• icelancer 13y ago
  I feel the exact opposite. ML is a huge part of my job (or was), and I topped out at Discrete Math 2 or so, finishing up a few courses of the usual Calculus but never anything as esoteric as Real Analysis or the like. (I think I finished up some Linear Algebra.) Anyway, my math is not that great and I'm way out of practice when it comes to theoretical work. Not only that, I suck at algorithm design and I'm at best a moderately decent developer from a theory standpoint. But applied machine learning came natural, because I have a strong grasp of frequentist and Bayesian statistics as well as solving the problems backwards. If you try to apply ML or any other "exotic" solution by starting with "oh man here is this awesome technique," you will definitely be frustrated; I know that feeling all too well. ML is just another tool in the shed. With practice, you start to understand where simple linear regression techniques are useful and when more heavy-duty methods are applicable. Keep solving real world problems rather than trying to look for stuff to hit with your new hammer (ML in this case) and I bet you'll come around a lot quicker.
• ajtulloch 13y ago
  My take on this is somewhat different - my background is pure mathematics in my undergraduate degree, then applied machine learning at Facebook (ads optimization) and GS (algo trading), and currently at graduate school in ML. My take is that 'applying' machine learning is much easier than one would think, but 'understanding' machine learning is _much_ harder than one would think. I imagine most people start with an application - that's why they're interested in the first place. Applying an ML algorithm to a given dataset is trivial - only a few lines in R/Python with scikits-learn, etc. There's even a handy cheat-sheet [1] to tell you which algorithm to use. Your regression model spits out a few parameters, some estimates of quality of fit, maybe some pretty graphs, and you can call it a day. The next stage is implementation - up until now, the black box has been spitting out parameters, and you want to understand what these numbers actually are and how they are determined. This means getting your hands dirty with the nitty-gritty of ML algorithms - understanding the algorithms statistical basis, understanding the computational performance of the algorithms (as you alluded to), and reading textbooks/journal articles. Many of these algorithms have a simple 'kernel' of an idea that can be obscured by a lot of mathematical sophistication - For SVM, the idea of finding a plane that separates your data as much as possible, logistic regression is just finding the line of best fit in logit space, k-means is just of repeatedly assigning points to clusters then updating clusters, etc. It's not _that_ difficult to get working implementations of these algorithms (although debugging an incorrect implementation can be frustrating). You can implement simple versions of gradient boosting machines - a theoretically sophisticated technique that often works very well in practice - in around 100 lines of Python [2] or Go [3]. You can get implement Hopfield networks (a basic neural network for unsupervised learning) in Julia [4] and Haskell [5] without much trouble either. I think where a lot of the value from spending 3-5+ years in graduate school comes in when you an off-the-shelf technique isn't cutting it - maybe you're overfitting, you want to play with hyperparameters, your datasets are scaling faster than the off-the-shelf implementations can deal with them, etc. Solving these kind of problems is where a lot of the mathematical/statistical sophistication can come into play - Bayesian hyperparameter optimization of existing ML techniques improved on state-of-the-art in image processing [6], understanding _why_ the LASSO works (and understanding when it doesn't) can mean having to understand the distribution of eigenvalues of random Gaussian matrices [7]. That's saying nothing of figuring out how to take an existing algorithm and scaling it out in parallel across 1000+ machines [8], which can mean using some deep convex optimization theory [9] to justify your approximations and prove that the approximated model you learn is well-behaved, etc. I suspect that some of the gulf between ML in practice and theory arises from the differing goals in the two parties. If you just want an parameter estimation, then a lot of the rigour will seem useless, and if you want to understand a procedure deeply, you can't do away with rigour. Take the k-means paper used in the article. In the screenshot, the paper's author is presenting results of the flavour > Under weak conditions on the data, as the number of iterations increases, regardless of the random choices made, the k-means clustering tends towards a good clustering of the data. Depending on your viewpoint, this result is either obvious, non-obvious, or irrelevant. To prove this needs (at a glance) theorems from topology, measure theory, martingale convergence, etc. I suppose some people find that fascinating, and want to understand _why_ this is true, and what 'good', 'tends', and 'weak' _mean_ in the above statement - and some people just find this irrelevant to their goal of finding similar blog articles. Both viewpoints are reasonable, but I suspect this divergence is the cause of the articles like these. Applying machine learning isn't that hard, but _understanding_ it is. [1]: https://lh3.ggpht.com/-ME24ePzpzIM/UQLWTwurfXI/AAAAAAAAANw/W3EETIroA80/s1600/drop_shadows_background.png https://lh3.ggpht.com/-ME24ePzpzIM/UQLWTwurfXI/AAAAAAAAANw/W... [2]: http://www.trungh.com/2013/04/a-short-introduction-to-gradient-boosting-machine-in-python/ http://www.trungh.com/2013/04/a-short-introduction-to-gradie... [3]: https://github.com/ajtulloch/decisiontrees https://github.com/ajtulloch/decisiontrees [4]: https://github.com/johnmyleswhite/HopfieldNets.jl https://github.com/johnmyleswhite/HopfieldNets.jl [5]: https://github.com/ajtulloch/hopfield-networks https://github.com/ajtulloch/hopfield-networks [6]: http://arxiv.org/pdf/1206.2944.pdf http://arxiv.org/pdf/1206.2944.pdf [7]: http://arxiv.org/pdf/math/0410542v3.pdf http://arxiv.org/pdf/math/0410542v3.pdf [8]: http://books.nips.cc/papers/files/nips25/NIPS2012_0598.pdf http://books.nips.cc/papers/files/nips25/NIPS2012_0598.pdf [9]: http://www.cs.berkeley.edu/~jduchi/projects/DuchiHaSi10.pdf http://www.cs.berkeley.edu/~jduchi/projects/DuchiHaSi10.pdf