3 ms·

In my experience no amount of reading or watching videos can really give you a sense of how deep learning works, what it’s good and bad at, and how it both does

In my experience no amount of reading or watching videos can really give you a sense of how deep learning works, what it’s good and bad at, and how it both does and doesn’t feel like magic. The only thing that works is coding something up in Torch or similar and trying it out.

aaron-santos 6y ago
I have a similar experience. For the last 10 months I've had the opportunity to use Keras at work. While reading deep learning articles prior to that gave me an idea of what I needed to know, it was no substitute for experience. In a lot of ways it parallels learning monads. It is something that has to be experienced. As an aside we're now at peak VAE tutorials in the same way we hit peak monad tutorials a few years back.
kthejoker2 6y ago
I mean .. I disagree? It's certainly simple to give "a sense" of how deep learning works - 1) translate real-world states and behaviors into data structures (typically multi-dimensional arrays); 2) use (relatively straightforward) math to find patterns in the large matrices; 3) assume those patterns are representative of the real-world states and behaviors it translated; 4) produces a model which "works in reverse" i.e. uses the patterns it found to generate real-world states and behaviors, particularly states and behaviors it has never translated or seen before. And therefore what it's good at - math, finding patterns - are simple to get a sense of, what it's bad at - assuming the patterns it's found are representative - is simple to understand, and why it feels like magic - the speed at which it can acquire a skill, that it's trained solely through observation - and why it doesn't feel like magic - it's just math and pattern-matching, things we teach in preschool. Every new breakthrough in AI/DL research is either "more math", "same math but faster", "more patterns", "better patterns" ... Always good to read and see how very smart people solved problems, but the underlying principles are really simple. What's way harder to wrap your head around is the distributed compute and data pipelines and hardware architectures required to support the scales at which these simple mathematical pattern-matching algorithms need to operate to be useful.

aaron-santos 6y ago

I have a similar experience. For the last 10 months I've had the opportunity to use Keras at work. While reading deep learning articles prior to that gave me an idea of what I needed to know, it was no substitute for experience. In a lot of ways it parallels learning monads. It is something that has to be experienced. As an aside we're now at peak VAE tutorials in the same way we hit peak monad tutorials a few years back.

kthejoker2 6y ago

I mean .. I disagree? It's certainly simple to give "a sense" of how deep learning works - 1) translate real-world states and behaviors into data structures (typically multi-dimensional arrays); 2) use (relatively straightforward) math to find patterns in the large matrices; 3) assume those patterns are representative of the real-world states and behaviors it translated; 4) produces a model which "works in reverse" i.e. uses the patterns it found to generate real-world states and behaviors, particularly states and behaviors it has never translated or seen before. And therefore what it's good at - math, finding patterns - are simple to get a sense of, what it's bad at - assuming the patterns it's found are representative - is simple to understand, and why it feels like magic - the speed at which it can acquire a skill, that it's trained solely through observation - and why it doesn't feel like magic - it's just math and pattern-matching, things we teach in preschool. Every new breakthrough in AI/DL research is either "more math", "same math but faster", "more patterns", "better patterns" ... Always good to read and see how very smart people solved problems, but the underlying principles are really simple. What's way harder to wrap your head around is the distributed compute and data pipelines and hardware architectures required to support the scales at which these simple mathematical pattern-matching algorithms need to operate to be useful.