5 ms·
A Brief Introduction to Machine Learning for Engineers (2017)
- tw1010 9y agoThis makes a good case for not learning machine learning three years ago. If you wait long enough for new paradigms to coagulate, you can use your energy more wisely during the chaotic transient period (e.g. by spending your effort on time-invariant parts of human knowledge, such as gaining a deeper understanding of parts of pure mathematics, or even philosophy or history), while you wait for a new set of fundamentals to converge, and really solid pedagogical explanations to surface. The same could be said about skipping the earlier period of javascript frameworks. Had you used your time to learn things that are still useful today, and will remain useful decades in the future, while waiting for the industry to jump here and there, eventually converging to what seems to be a fairly robust steady state (react), then you'd probably be better off today than if you followed along with the hype cycle month after month. A counter argument is that you might miss out on a lot of the monetary rewards that comes from learning the much desired skill that others don't have the time or will power to tackle in the period where doing so involves a lot of friction.
- metasyn 9y agoLooking through the TOC - It seems like most of this was in just as a "fairly robust steady state" as it was three years ago? Was there some part of this that struck you as having solidified in the last few years?
- briga 9y agoA lot of the recent successes in ML haven't been based on new breakthroughs in theory. For instance deep neural networks have been studied for decades now. Same with the reinforcement learning algorithms behind advances like AlphaGo. So I don't think skipping ahead five years will necessarily help you. That said, there was a lot of AI work with hand-crafted features and expert systems that has no basically been rendered obsolete by deep learning. But advances generally don't emerge out of a vacuum, and it helps to have some background knowledge of previous work to have a good idea of what has and hasn't worked in the past.
- cscurmudgeon 9y agoWe have replaced hand-crafted features with hand-crafted architectures. While performance might be important, at least the features were easy to understand by non experts. :)
- halflings 9y agoI find it's much easier to understand why 3 CNN layers applied on the raw image can learn convolutions that are relevant to solving my task... than it is to understand what is being done by "artisanal hand-engineered authentic patches" with a couple obscure dimensionality reduction algorithms thrown in, and an SVM on top. Expert systems were also incredibly hard to build and debug, and weren't nearly as useful as ML systems are nowadays.
- vikiomega9 9y ago> A counter argument is that you might miss out on a lot of the monetary rewards that comes from learning the much desired skill that others don't have the time or will power to tackle in the period where doing so involves a lot of friction Perhaps it can be argued that one is not in the position to take advantage of such situations. Switching jobs is not necessarily cheap
- FullMtlAlcoholc 9y agoAnother counter-argument is you're still learning about frameworks in general which should help you pick up new ones quicker or provide a more comprehensive base of knowledge to draw upon when developing your own.
- nabla9 9y agoProgramming frameworks have lots of stuff that becomes obsolete over time. You don't really learn complex, open-ended and expanding mathematical subject like machine learning like you describe. There is no book you just read and then you understand and now it's behind you. You are constantly making new connections and deepening your understanding when you learn more. Ten years is small time for things to start to sink in and make a connected whole in the mind.
- wirrbel 9y ago> This makes a good case for not learning machine learning three years ago. Have you looked at the monograph at all? This could have been written 5 to 10 years ago, 15 years ago, maybe a few sections would have looked a little different. In fact, I like to recommend Mitchell's Machine Learning book (I think it was written in the 90s) as an introduction to people with a serious interest. There currently is a lot of hype going on for machine learning algorithms, because we see good progress in things like computer vision / pattern recognition. This kicks of a marketing machinery that really blurs the reality. In reality, we have an established body of methods and modelling techniques that are sufficient, because the available data is the bottleneck for prediction quality. The actual challenge is to come up with a valuable business proposition, not necessarily to build the predictive model.
- dotancohen 9y ago> Have you looked at the monograph at all? This could have been written 5 to 10 years ago The counterpoint is that a _lot_ could have been written 5 to 10 years ago, and only 10% of it would still be relevant. This is the relevant 10%.
- wirrbel 9y agoLooking at the table of contents, it is a subset of PRML [Bishop 2006]. And I suppose it is more than 10% PRML. And the things contained in PRML but not in the monograph are still relevant I'd say (Kalman filters, ...). [Bishop 2006] http://www.springer.com/de/book/9780387310732 http://www.springer.com/de/book/9780387310732
- bitL 9y agoI'd like to point out the fact that there is a split within Machine Learning into "classical" and "Deep Learning"-based one. While Deep Learning are just neural networks with Big Data on GPU, they obliterated many areas of classical ML. What you really want to learn is Deep (+Reinforcement) Learning, or what is coming known as differentiable programming. For classical ML, learn how to use Spark's MLlib and take Ng's Coursera course, that might be all you need from practical point of view.
- LouisSayers 9y agoFunnily enough this is a strategy I’ve been using for years. At University I used to wait a while for others to complete their projects and then pick their brains about how they went about solving them. Now in industry I don’t bother learning new tech until I start seeing those skills show up on job sites or if I have a very particular need that requires a particular technology as part of the solution. This strategy has treated me well but I believe has only done so because not everyone takes this approach. I’m also grateful for the time and effort others put in to allow me to essentially freeload off of their hard work. Every now and then I do find something I can be truly passionate about to help others do the same.
- ardillaroja 9y agoRelevant xkcd (https://xkcd.com/989/ https://xkcd.com/989/)
- draw_down 9y agoIf anything this comic is about how highly engineers regard themselves. If they all go to sleep for a long time nothing will advance. Yeah, right.
- jimothyhalpert7 9y agoThis is my reasoning about blockchain. It was the first time I realised that just because I'm disposed to understand something better than the average person, doesn't mean I have to. Even if that something will be part of everyday life for every person. Society will just take care of packaging it into a familiar interface, e.g. credit card. I then however realised that it would be a shame to let my advantage go to waste. So I use this: > a lot of the monetary rewards to partially motivate myself.
- kmax12 9y agoThis appears to be an thorough overview of machine learning. Even though many bases are covered, I wish there was more on how to create or select features for machine learning in a systematic way. Feature extraction gets 1 paragraph, but feature engineering and selection aren't mentioned much in the 200+ pages! I don't have a Phd in machine learning, but I have spent many years using it as a tool to solve problems. While the details here can get you a long way, without understand feature engineering or feature selection, you will have a hard time building accurate models. For any engineers looking for more on feature engineering after reading this, I maintain an open source library for automated feature engineering called Featuretools (https://github.com/featuretools/featuretools https://github.com/featuretools/featuretools). We also have demos on our website (https://www.featuretools.com/demos https://www.featuretools.com/demos) if you want to see it in action.
- bistro17 9y agoI was wondering how featuretools differs from https://github.com/AxeldeRomblay/MLBox https://github.com/AxeldeRomblay/MLBox https://github.com/crawles/automl_service https://github.com/crawles/automl_service and the proprietary and newly launched driverless ai (from h2o)
- kmax12 9y agoFeaturetools focuses on handling data with relational structure and timestamps. Here's an example to explain those two key points. Imagine you have a relational database from a retail store with tables for customers, transactions, products, and stores. Featuretools can make a feature matrix for any entity in the database using an algorithm called Deep Feature Synthesis. We wrote a blog post about it here: https://www.featurelabs.com/blog/deep-feature-synthesis/ https://www.featurelabs.com/blog/deep-feature-synthesis/. Basically, it tries to stack dataset-agnostic "feature primitives" to construct features similar to what human data scientists would create. This means that a data scientist can go from building models about their customers to models about their stores in one line of code. One aspect worth highlighting is that Featuretools can be extended with custom primitives to expand the set of features in can produce. As the repo of primitives grows, everyone in the community benefits because primitives aren't tied to a specific dataset or use case. Some of our demos highlight this functionality to increase scores on the Kaggle leaderboards. Featuretools is good at handling time. When performing feature engineering on completely raw data it is important not to mix up time. When your data is timestamped, you can tell Featuretools to create features at any point in time and it automatically slices the data for you (even across relationships between tables!). You want to avoid situations similar to training a machine learning model on stock market data from 2017, testing that it works on data from 2016, and then deploying it and expecting to make money in 2018. You can read more about how featuretools handle time here: https://docs.featuretools.com/automated_feature_engineering/handling_time.html https://docs.featuretools.com/automated_feature_engineering/...)
- cpeterso 9y ago"Applying Machine Learning Like a Responsible Adult", a 30 minute talk from GDC 2017, is a nice introduction to common machine learning approaches and pitfalls for game developers: https://www.youtube.com/watch?v=RLsKzkxWpK8 https://www.youtube.com/watch?v=RLsKzkxWpK8
- Myrmornis 9y agoDocuments like this (like traditional books) on technical subjects need to be read very carefully, requiring a time commitment that is large relative to the length of a human life or career. Also, they have a huge amount of overlap with other similar documents, so critical choices must be made about what to try to read. It would be nice to have a high-quality and widely-used set of community ratings for such documents. E.g. a place where - new documents can be added - they are categorized (automatically should be doable) according to subject matter, level, etc - some sort of community voting system (perhaps augmented by automated scoring by well-established predictors) scores each document for its utility/recommendability, in each of the subject areas that it covers. Does anything like that exist for arXiv? Do people put general expository material on arXiv? (E.g. lecture notes, textbooks, etc).
- cosmic_ape 9y agoFunny that in the abstract it aims specifically at electrical engineers, since electrical engineers do machine learning for about six decades now, under different names. Since the invention of the Wiener filter, the Kalman filter, etc...
- eddd 9y agoI wish someone said to me a year ago (when I started ML) clearly at the beginning: "Machine Learning is about MATH, deal with it". It'd be much easier.
- verletx64 9y agoI think it's very contentious to some people (though I agree that at this stage, ML == mathematics). I've read quite a few people on the internet who will swear up and down you don't need the mathematics to apply basic models to business problems. I disagree, and I kind of find it weird to divorce the mathematics from it.
- wirrbel 9y agoyes and no. Of course it is absurd to claim you don't need math. On the other hand, I think it is sometimes exaggerated. You'll see this tutorials, that are thorough linear algebra primers and claim that you need it for ML, when in fact you'd only need it to get a thorough understanding of the inner workings. On the other hand, I have seen highly educated math experts pretty much fail to understand, that a L2 norm isn't the best loss function for a business context where a deviation from the truth actually means linear costs. So I'd argue being able to map business problems into the shallow math domain is much more important, than mastering the deep math domain.
- kingkongjaffa 9y agoI know ML is the new hotness but engineers (proper engineers) have used it for years in basic areas for condition monitoring.. - hooking up sensor input from your oil pumps to a neural network to understand statistics of your population to predict damage. https://en.wikipedia.org/wiki/Condition_monitoring https://en.wikipedia.org/wiki/Condition_monitoring
- amelius 9y agoI think the concept of Machine Learning clashes in many ways with the core values of an engineer. Engineering is about constructing new things, making them correct by design. Machine learning, on the other hand, is throwing data at a problem, waiting for data to be processed, and hoping that the resulting models make sense. While both useful, it requires such a different mindset, that I wonder if engineers would feel satisfied working on such solutions.
- skate22 9y agoML in practice (from my xp) is mostly getting and cleaning data. As far as training, testing, and deploying a model, to an engineer it's just more algorithms. While some algorithms may have more explainability than others, the engineer cares if they solve the business problem at hand.
- dotancohen 9y agoThe root of the word "engineer" means "to give birth to". In that sense, using Machine Learning to evolutionry give rise to new models is actually quite fitting.
- cup-of-tea 9y agoUsing machine learning is indeed more like science than engineering. After all it is just building a model based on data that you have seen. The advantage of using ML is the model can be arbitrarily complex and something you'd never be able to come up with manually. But that model is only a part of a larger project. To the engineer it seems no different to designing a solution in the context of accepted scientific theories. You can't engineer the theories, they are accepted based on evidence. But you can build the project around it.
- zild3d 9y ago>Engineering is about constructing new things Engineering is about solving problems under constraints. ML is a tool in the tool box to use against problems when the constraints are appropriate, similar to just about any algorithm
- joaovictortr 9y agoAnother very interesting resource on the subject is the book Introduction to Statistical Learning by Garet James et al [1]. The book introduces the foundational concepts of statistical learning (classification, regression, cross-validation) and algorithms such as support vector machines. It is also available on PDF at the website [1]. [1] http://www-bcf.usc.edu/~gareth/ISL/ http://www-bcf.usc.edu/~gareth/ISL/
- udev 9y agoFunny how the author pushes the language to extremes: - title: a "brief" introduction (it's 206 pages!) - chapter 1: a "gentle" introduction through Linear Regression, where gentle means that the relationships and equations are provided in all their notational beauty, but without the motivation or meaning part.
- martinjoshen 9y agohttps://sites.google.com/site/gujjubhaimostwantedgujaratihd/ https://sites.google.com/site/gujjubhaimostwantedgujaratihd/