5 ms·

This makes a good case for not learning machine learning three years ago. If you wait long enough for new paradigms to coagulate, you can use your energy more w

by tw1010 9y ago

This makes a good case for not learning machine learning three years ago. If you wait long enough for new paradigms to coagulate, you can use your energy more wisely during the chaotic transient period (e.g. by spending your effort on time-invariant parts of human knowledge, such as gaining a deeper understanding of parts of pure mathematics, or even philosophy or history), while you wait for a new set of fundamentals to converge, and really solid pedagogical explanations to surface. The same could be said about skipping the earlier period of javascript frameworks. Had you used your time to learn things that are still useful today, and will remain useful decades in the future, while waiting for the industry to jump here and there, eventually converging to what seems to be a fairly robust steady state (react), then you'd probably be better off today than if you followed along with the hype cycle month after month. A counter argument is that you might miss out on a lot of the monetary rewards that comes from learning the much desired skill that others don't have the time or will power to tackle in the period where doing so involves a lot of friction.

metasyn 9y ago
Looking through the TOC - It seems like most of this was in just as a "fairly robust steady state" as it was three years ago? Was there some part of this that struck you as having solidified in the last few years?
briga 9y ago
A lot of the recent successes in ML haven't been based on new breakthroughs in theory. For instance deep neural networks have been studied for decades now. Same with the reinforcement learning algorithms behind advances like AlphaGo. So I don't think skipping ahead five years will necessarily help you. That said, there was a lot of AI work with hand-crafted features and expert systems that has no basically been rendered obsolete by deep learning. But advances generally don't emerge out of a vacuum, and it helps to have some background knowledge of previous work to have a good idea of what has and hasn't worked in the past.
- cscurmudgeon 9y ago
  We have replaced hand-crafted features with hand-crafted architectures. While performance might be important, at least the features were easy to understand by non experts. :)
  - halflings 9y ago
    I find it's much easier to understand why 3 CNN layers applied on the raw image can learn convolutions that are relevant to solving my task... than it is to understand what is being done by "artisanal hand-engineered authentic patches" with a couple obscure dimensionality reduction algorithms thrown in, and an SVM on top. Expert systems were also incredibly hard to build and debug, and weren't nearly as useful as ML systems are nowadays.
vikiomega9 9y ago
> A counter argument is that you might miss out on a lot of the monetary rewards that comes from learning the much desired skill that others don't have the time or will power to tackle in the period where doing so involves a lot of friction Perhaps it can be argued that one is not in the position to take advantage of such situations. Switching jobs is not necessarily cheap
FullMtlAlcoholc 9y ago
Another counter-argument is you're still learning about frameworks in general which should help you pick up new ones quicker or provide a more comprehensive base of knowledge to draw upon when developing your own.
nabla9 9y ago
Programming frameworks have lots of stuff that becomes obsolete over time. You don't really learn complex, open-ended and expanding mathematical subject like machine learning like you describe. There is no book you just read and then you understand and now it's behind you. You are constantly making new connections and deepening your understanding when you learn more. Ten years is small time for things to start to sink in and make a connected whole in the mind.
wirrbel 9y ago
> This makes a good case for not learning machine learning three years ago. Have you looked at the monograph at all? This could have been written 5 to 10 years ago, 15 years ago, maybe a few sections would have looked a little different. In fact, I like to recommend Mitchell's Machine Learning book (I think it was written in the 90s) as an introduction to people with a serious interest. There currently is a lot of hype going on for machine learning algorithms, because we see good progress in things like computer vision / pattern recognition. This kicks of a marketing machinery that really blurs the reality. In reality, we have an established body of methods and modelling techniques that are sufficient, because the available data is the bottleneck for prediction quality. The actual challenge is to come up with a valuable business proposition, not necessarily to build the predictive model.
- dotancohen 9y ago
  > Have you looked at the monograph at all? This could have been written 5 to 10 years ago The counterpoint is that a _lot_ could have been written 5 to 10 years ago, and only 10% of it would still be relevant. This is the relevant 10%.
  - wirrbel 9y ago
    Looking at the table of contents, it is a subset of PRML [Bishop 2006]. And I suppose it is more than 10% PRML. And the things contained in PRML but not in the monograph are still relevant I'd say (Kalman filters, ...). [Bishop 2006] http://www.springer.com/de/book/9780387310732 http://www.springer.com/de/book/9780387310732
- bitL 9y ago
  I'd like to point out the fact that there is a split within Machine Learning into "classical" and "Deep Learning"-based one. While Deep Learning are just neural networks with Big Data on GPU, they obliterated many areas of classical ML. What you really want to learn is Deep (+Reinforcement) Learning, or what is coming known as differentiable programming. For classical ML, learn how to use Spark's MLlib and take Ng's Coursera course, that might be all you need from practical point of view.
LouisSayers 9y ago
Funnily enough this is a strategy I’ve been using for years. At University I used to wait a while for others to complete their projects and then pick their brains about how they went about solving them. Now in industry I don’t bother learning new tech until I start seeing those skills show up on job sites or if I have a very particular need that requires a particular technology as part of the solution. This strategy has treated me well but I believe has only done so because not everyone takes this approach. I’m also grateful for the time and effort others put in to allow me to essentially freeload off of their hard work. Every now and then I do find something I can be truly passionate about to help others do the same.
- ardillaroja 9y ago
  Relevant xkcd (https://xkcd.com/989/ https://xkcd.com/989/)
  - draw_down 9y ago
    If anything this comic is about how highly engineers regard themselves. If they all go to sleep for a long time nothing will advance. Yeah, right.
jimothyhalpert7 9y ago
This is my reasoning about blockchain. It was the first time I realised that just because I'm disposed to understand something better than the average person, doesn't mean I have to. Even if that something will be part of everyday life for every person. Society will just take care of packaging it into a familiar interface, e.g. credit card. I then however realised that it would be a shame to let my advantage go to waste. So I use this: > a lot of the monetary rewards to partially motivate myself.

by tw1010 9y ago

metasyn 9y ago

Looking through the TOC - It seems like most of this was in just as a "fairly robust steady state" as it was three years ago? Was there some part of this that struck you as having solidified in the last few years?

briga 9y ago

A lot of the recent successes in ML haven't been based on new breakthroughs in theory. For instance deep neural networks have been studied for decades now. Same with the reinforcement learning algorithms behind advances like AlphaGo. So I don't think skipping ahead five years will necessarily help you. That said, there was a lot of AI work with hand-crafted features and expert systems that has no basically been rendered obsolete by deep learning. But advances generally don't emerge out of a vacuum, and it helps to have some background knowledge of previous work to have a good idea of what has and hasn't worked in the past.

cscurmudgeon 9y ago
We have replaced hand-crafted features with hand-crafted architectures. While performance might be important, at least the features were easy to understand by non experts. :)
- halflings 9y ago
  I find it's much easier to understand why 3 CNN layers applied on the raw image can learn convolutions that are relevant to solving my task... than it is to understand what is being done by "artisanal hand-engineered authentic patches" with a couple obscure dimensionality reduction algorithms thrown in, and an SVM on top. Expert systems were also incredibly hard to build and debug, and weren't nearly as useful as ML systems are nowadays.

vikiomega9 9y ago

> A counter argument is that you might miss out on a lot of the monetary rewards that comes from learning the much desired skill that others don't have the time or will power to tackle in the period where doing so involves a lot of friction Perhaps it can be argued that one is not in the position to take advantage of such situations. Switching jobs is not necessarily cheap

FullMtlAlcoholc 9y ago

Another counter-argument is you're still learning about frameworks in general which should help you pick up new ones quicker or provide a more comprehensive base of knowledge to draw upon when developing your own.

nabla9 9y ago

Programming frameworks have lots of stuff that becomes obsolete over time. You don't really learn complex, open-ended and expanding mathematical subject like machine learning like you describe. There is no book you just read and then you understand and now it's behind you. You are constantly making new connections and deepening your understanding when you learn more. Ten years is small time for things to start to sink in and make a connected whole in the mind.

wirrbel 9y ago

> This makes a good case for not learning machine learning three years ago. Have you looked at the monograph at all? This could have been written 5 to 10 years ago, 15 years ago, maybe a few sections would have looked a little different. In fact, I like to recommend Mitchell's Machine Learning book (I think it was written in the 90s) as an introduction to people with a serious interest. There currently is a lot of hype going on for machine learning algorithms, because we see good progress in things like computer vision / pattern recognition. This kicks of a marketing machinery that really blurs the reality. In reality, we have an established body of methods and modelling techniques that are sufficient, because the available data is the bottleneck for prediction quality. The actual challenge is to come up with a valuable business proposition, not necessarily to build the predictive model.

dotancohen 9y ago
> Have you looked at the monograph at all? This could have been written 5 to 10 years ago The counterpoint is that a _lot_ could have been written 5 to 10 years ago, and only 10% of it would still be relevant. This is the relevant 10%.
- wirrbel 9y ago
  Looking at the table of contents, it is a subset of PRML [Bishop 2006]. And I suppose it is more than 10% PRML. And the things contained in PRML but not in the monograph are still relevant I'd say (Kalman filters, ...). [Bishop 2006] http://www.springer.com/de/book/9780387310732 http://www.springer.com/de/book/9780387310732
bitL 9y ago
I'd like to point out the fact that there is a split within Machine Learning into "classical" and "Deep Learning"-based one. While Deep Learning are just neural networks with Big Data on GPU, they obliterated many areas of classical ML. What you really want to learn is Deep (+Reinforcement) Learning, or what is coming known as differentiable programming. For classical ML, learn how to use Spark's MLlib and take Ng's Coursera course, that might be all you need from practical point of view.

LouisSayers 9y ago

Funnily enough this is a strategy I’ve been using for years. At University I used to wait a while for others to complete their projects and then pick their brains about how they went about solving them. Now in industry I don’t bother learning new tech until I start seeing those skills show up on job sites or if I have a very particular need that requires a particular technology as part of the solution. This strategy has treated me well but I believe has only done so because not everyone takes this approach. I’m also grateful for the time and effort others put in to allow me to essentially freeload off of their hard work. Every now and then I do find something I can be truly passionate about to help others do the same.

ardillaroja 9y ago
Relevant xkcd (https://xkcd.com/989/ https://xkcd.com/989/)
- draw_down 9y ago
  If anything this comic is about how highly engineers regard themselves. If they all go to sleep for a long time nothing will advance. Yeah, right.

jimothyhalpert7 9y ago

This is my reasoning about blockchain. It was the first time I realised that just because I'm disposed to understand something better than the average person, doesn't mean I have to. Even if that something will be part of everyday life for every person. Society will just take care of packaging it into a familiar interface, e.g. credit card. I then however realised that it would be a shame to let my advantage go to waste. So I use this: > a lot of the monetary rewards to partially motivate myself.