5 ms·
This might be indeed true for deep learning in its current shape. However, in the long run I think we will see more models that (1) are more composable and (2)
by wirrbel 9y ago
This might be indeed true for deep learning in its current shape. However, in the long run I think we will see more models that (1) are more composable and (2) better approaches to engineering such models, however, these, engineered models will look a bit different than what your research scientist throws over the fence today.
At the moment we are in a phase were, to stick to the optics metaphor, we stack up lenses until we see the object on the screen. This means we end up with models that are sprawling, instead of having models that were engineered.
Another trend that seems to start in deep learning is that layers become more constrained. I expect, that in 20 years, we will see much more constrained models and much generative models.
- m_ke 9y ago1. deep learning models are extremely composable 2. hopefully with time we'll have better approaches to engineer all things that are engineered No, at the moment we go for the biggest and shiniest lens that we can get our hands on and hope that it's capable enough to tackle our problem. If it is we can waste time designing a smaller, more constrained, lens to ship to consumers.
- taeric 9y agoI'm curious if you two are going to be talking past each other with the first point. Any chance I could get you both to explore what you mean by composable?
- m_ke 9y agoThese days any method that uses gradient descent to optimize a computational graph gets branded as deep learning. It's a very general paradigm that allows for almost any composition of functions as long as it's differentiable. If that's not composable then I don't know what is. There's a reason why Lecun wanted to rebrand deep learning as differentiable programming. https://www.facebook.com/yann.lecun/posts/10155003011462143 https://www.facebook.com/yann.lecun/posts/10155003011462143 I'm not sure what wirrbel meant.
- taeric 9y agoMy guess for what they meant was that you can't compose the trained models. For example, a classifier that tells you cat or not can't be used with one that says running or not to get running cat. The benefit being that you could put together more "off the shelf" models into products. Instead, you have to train up pretty much everything from the ground. And we compare against others doing the same. @wirrbel, that accurate?
- taeric 9y agoWhat makes you think this? So far, I don't think any field actually fits this description. (As such, my personal priors for this are quite low.) Are there fields where this is an apt description?