3 ms·

It was kinda hard for me to understand the basics of deep learning. A lot of formal courses makes a lot of assumptions. I ended up watching the deep learning s

It was kinda hard for me to understand the basics of deep learning. A lot of formal courses makes a lot of assumptions. I ended up watching the deep learning series by 3blue1brown, reading the book it was based on. Watching lots of linear algebra and calculus videos. Then taking the deep learning specialization program on Coursera. This process took 2 months, it taught me the basics. Then I built my own project and realized there were lots wholes in my knowledge base. I found an old post on hn that recommended deeplizard on YouTube, that was an excellent recommendation and intro to pytorch. One thing to keep in mind is that as you are learning, new methods are being invented. Edit: typos

iamcreasy 5y ago
> I ended up watching the deep learning series by 3blue1brown, reading the book it was based on. What book you are referring to?
- firefoxd 5y ago
  This one http://neuralnetworksanddeeplearning.com http://neuralnetworksanddeeplearning.com It goes much deeper then the videos. Also it is free.
  - max_ 5y ago
    That book is excellent. It's the only one that I can recommend to beginners.
  - atum47 5y ago
    Great
mdp2021 5y ago
Sorry, speaking of web-based books about Deep Learning, I remember one in which the figures allowed changing parameters and seeing the results - I cannot remember the source or author, can anyone? I confused it with the one from Michael Nielsen...
amenod 5y ago
I went through multiple resources on the net until it clicked for me. I guess each book / video / blog post gives you the bits you are ready for, and when enough of these bits accumulate you understand it. Looking back, neural networks are not that difficult to grasp, but there are many important details that one should understand. What I learned is that it is very important to test intermediate results thoroughly. Nets will gladly learn from garbage if that is what you (unknowingly) feed them, and they might even produce some results - not at acceptable level though. I didn't continue working in this field, but I'm happy I learned it (and used it in a project too) and would recommend that to any developer. ML really moves the frontier of what we can achieve in software much further.
euske 5y ago
Here's my attempt to explain the whole thing: Almost all modern AI is a search problem; We're just trying to find a clever way to search the problem space. Most supervised learning methods are a search problem for best programs (models) that explain the data; We're just trying to find a clever way to express the model and efficient algorithms to search the problem space. Many popular ML algorithms (NN, SVM, etc.) are an optimization problem; you just want to find parameters to maximize some performance metrics. But you don't want to search all the parameters exhaustively. You want something that can be improved incrementally over time. NN is popular because it's flexible (can have thousands of inputs/outputs) and can be built incrementally. The secret sauce is its performance metrics (loss function) being differentiable. This way, you can incrementally improve the model via gradient descent. NN is popular also because in theory you can stack as many layers as you want, giving it more power to learn complex relationships. It's applicable to many tasks. CNN/RNN/LTSM/Transformer are just variations of these. They have different structures, but all operating on the same principle: differentiable loss function and iterative training methods.

iamcreasy 5y ago

> I ended up watching the deep learning series by 3blue1brown, reading the book it was based on. What book you are referring to?

firefoxd 5y ago
This one http://neuralnetworksanddeeplearning.com http://neuralnetworksanddeeplearning.com It goes much deeper then the videos. Also it is free.
- max_ 5y ago
  That book is excellent. It's the only one that I can recommend to beginners.
- atum47 5y ago
  Great

mdp2021 5y ago

Sorry, speaking of web-based books about Deep Learning, I remember one in which the figures allowed changing parameters and seeing the results - I cannot remember the source or author, can anyone? I confused it with the one from Michael Nielsen...

amenod 5y ago

I went through multiple resources on the net until it clicked for me. I guess each book / video / blog post gives you the bits you are ready for, and when enough of these bits accumulate you understand it. Looking back, neural networks are not that difficult to grasp, but there are many important details that one should understand. What I learned is that it is very important to test intermediate results thoroughly. Nets will gladly learn from garbage if that is what you (unknowingly) feed them, and they might even produce some results - not at acceptable level though. I didn't continue working in this field, but I'm happy I learned it (and used it in a project too) and would recommend that to any developer. ML really moves the frontier of what we can achieve in software much further.

euske 5y ago

Here's my attempt to explain the whole thing: Almost all modern AI is a search problem; We're just trying to find a clever way to search the problem space. Most supervised learning methods are a search problem for best programs (models) that explain the data; We're just trying to find a clever way to express the model and efficient algorithms to search the problem space. Many popular ML algorithms (NN, SVM, etc.) are an optimization problem; you just want to find parameters to maximize some performance metrics. But you don't want to search all the parameters exhaustively. You want something that can be improved incrementally over time. NN is popular because it's flexible (can have thousands of inputs/outputs) and can be built incrementally. The secret sauce is its performance metrics (loss function) being differentiable. This way, you can incrementally improve the model via gradient descent. NN is popular also because in theory you can stack as many layers as you want, giving it more power to learn complex relationships. It's applicable to many tasks. CNN/RNN/LTSM/Transformer are just variations of these. They have different structures, but all operating on the same principle: differentiable loss function and iterative training methods.