5 ms·

I'd clarify that there is a specific delusion that any data scientist straight out of some sort of online degree program can go toe to toe with the likes of And

I'd clarify that there is a specific delusion that any data scientist straight out of some sort of online degree program can go toe to toe with the likes of Andrej Karpathy or David Silver with the power of "teh durp lurnins'." And the predictable disappointment arising from the craptastic shovelware they create is what's finally creating the long overdue disappointment. Further, I have repeatedly heard people who should know better, with very fancy advanced degrees, chant variants of "Deep Learning gets better with more data" and/or "Deep Learning makes feature engineering obsolete" as if they are trying to convince everyone around them as well as themselves that these two fallacious assumptions are the revealed truth handed down to mere mortals by the 4 horsemen of the field. That said, if you put your ~10,000 hours into this, and keep up with the field, it's pretty impressive what high-dimensional classification and regression can do. Judea Pearl concurs: https://www.theatlantic.com/technology/archive/2018/05/machine-learning-is-stuck-on-asking-why/560675/ https://www.theatlantic.com/technology/archive/2018/05/machi... My personal (and admittedly biased) belief is that if you combine DL with GOFAI and/or simulation, you can indeed work magic. AlphaZero is strong evidence of that, no? And the author of the article in this thread is apparently attempting to do the same sort of thing for self-driving cars. I wouldn't call this part of the field irrational exuberance, I'd call it amazing.

bitL 8y ago
> Deep Learning makes feature engineering obsolete I think even if you avoid constructing features, you are basically doing a similar process where a single change in a hyper-parameter can have significant effects: - internal structure of a model (what types of blocks are you using and how do you connect them, what are they capable of together, how do gradients propagate?) - loss function (great results come only if you use a fitting loss function) - category weights (i.e. improving under-represented classes) - image/data augmentation (self-driving car won't work without significant augmentation at all) - properly set-up optimizer The good thing here is that you can automate optimization of these to a large extent if you have a cluster of machines and a way to orchestrate meta-optimization of slightly changed models. With feature engineering you just have to do all the work upfront, thinking what might be important, and often you just miss important parts of features :-(
- varelse 8y ago
  Yep, and in doing so, you just traded "feature engineering" for graph design and data prep, no? And that's my response to these sorts. And their usual response to me is to grumble that I don't know what I'm doing. I've started tuning them out of my existence because they seem to have nothing to contribute to it.
  - jacquesm 8y ago
    It's a huge difference in terms of the time invested to create something that performs well. Hand crafted feature engineering is better for some tasks but for quite a few of them automated methods perform very well indeed (at least, better than I expected).
- riku_iki 8y ago
  > if you have a cluster of machines and a way to orchestrate meta-optimization of slightly changed models curious, if there is any good quality open source project for this..
  - pizza 8y ago
    I'm not aware of actually written code/projects that does this, but try looking into neural architecture search, it should be useful https://github.com/markdtw/awesome-architecture-search https://github.com/markdtw/awesome-architecture-search

bitL 8y ago

> Deep Learning makes feature engineering obsolete I think even if you avoid constructing features, you are basically doing a similar process where a single change in a hyper-parameter can have significant effects: - internal structure of a model (what types of blocks are you using and how do you connect them, what are they capable of together, how do gradients propagate?) - loss function (great results come only if you use a fitting loss function) - category weights (i.e. improving under-represented classes) - image/data augmentation (self-driving car won't work without significant augmentation at all) - properly set-up optimizer The good thing here is that you can automate optimization of these to a large extent if you have a cluster of machines and a way to orchestrate meta-optimization of slightly changed models. With feature engineering you just have to do all the work upfront, thinking what might be important, and often you just miss important parts of features :-(

varelse 8y ago
Yep, and in doing so, you just traded "feature engineering" for graph design and data prep, no? And that's my response to these sorts. And their usual response to me is to grumble that I don't know what I'm doing. I've started tuning them out of my existence because they seem to have nothing to contribute to it.
- jacquesm 8y ago
  It's a huge difference in terms of the time invested to create something that performs well. Hand crafted feature engineering is better for some tasks but for quite a few of them automated methods perform very well indeed (at least, better than I expected).
riku_iki 8y ago
> if you have a cluster of machines and a way to orchestrate meta-optimization of slightly changed models curious, if there is any good quality open source project for this..
- pizza 8y ago
  I'm not aware of actually written code/projects that does this, but try looking into neural architecture search, it should be useful https://github.com/markdtw/awesome-architecture-search https://github.com/markdtw/awesome-architecture-search