6 ms·

Any recommendations on learning PyTorch. I see many people on Kaggle who use it.

matsemann 5y ago
Fast.ai is basically pytorch + loads of utility functions. By following the course/book, one to some extend learns what fastai does, and how it uses pytorch for practical stuff.
- carbocation 5y ago
  I have also had good success with FastAI as my initial hook into PyTorch. A brief post that shows just how thin the FastAI layer can be (if you want!) is here: https://muellerzr.github.io/fastblog/2021/02/14/Pytorchtofastai.html https://muellerzr.github.io/fastblog/2021/02/14/Pytorchtofas...
- kriro 5y ago
  I second this but once you are comfortable with fast.ai it's a good idea to reproduce some things in pure PyTorch eventually because it'll give you a deep understanding and appreciation for fast.ai as well.
- willj 5y ago
  One thing with fastai that annoyed me when I built a project with it was that the v1 and v2 APIs are totally different, and if you google or search stackoverflow for help with something, I found it more likely to stumble on answers for the v1 API than the v2 API. I also didn’t find their documentation super helpful for more than the most basic things (though not all documentation can be as amazing as scikit-learn).
- apohn 5y ago
  Am I allowed to have a contrarian opinion :) I went through the first version of FastAI (when it was Keras, torch?/tensorflow?) and forgot most of it because never did anything with Deep Learning. Then I did the FastAI V1 course again where they use FastAI library V2. I really liked the first version of the course because I felt like Jeremy did an awesome job of balancing understanding the guts of using a DL library with getting stuff done. It was a tough course, but I felt like I really understood things. I felt like the version with FastAI library V2 went too far into "Here are some commands you can use in the FastAI V2 library to to do this sexy thing with Deep Learning." I completed that course and really felt it should have been titled "A Course on the Fast AI V2 Library" I recently purchased "Deep Learning with PyTorch" by Eli Stevens. I've been working through this book and feel like it explains things a lot more. I'm haven't finished the book with it, but I do like it so far.
  - carom 5y ago
    I felt similarly about the fastai course. I'm doing the deeplearning.ai course (Andrew Ng) and I really like it. Warning though, it is bottom up, so you start by implementing neural nets with numpy to understand the internals, then it gets into TensorFlow. Just another resource if people are looking to learn deep learning and not pytorch in particular.
olavgg 5y ago
There are lot of tutorials on the PyTorch website where you can start playing around with several cool examples in many different categories. https://pytorch.org/tutorials/beginner/pytorch_with_examples.html https://pytorch.org/tutorials/beginner/pytorch_with_examples...
vpj 5y ago
We did a bunch of popular research paper implementations in PyTorch with notes (annotations); might be helpful. https://nn.labml.ai https://nn.labml.ai
- srvmshr 5y ago
  @vpj The only drawback I see is that much of the implementation is abstracted by your helper libraries. Not everyone wants to add an extra dependency layer. Otherwise the walkthroughs are super helpful.
  - deleted 5y ago
    [deleted]
  - vpj 5y ago
    Agreed. But it makes implementing and testing a lot easier for us. We try to minimize the dependencies as much as possible. Do you think it also comes in the way of understanding the paper and learning how to implement it?
    - srvmshr 5y ago
      Since the backbone of the implementation is packed away in the library import, I felt it didn't quite show the code & variable interaction well enough. Don't get me wrong. It is useful & concise like you mentioned. But your target audience is beginners & adopters, and it makes it no different from another framework such as Fastai (I have major gripes with them. It has a much bottled-in experience) To be true to walkthroughs, please consider designing helper functions rather than using your framework. Admittedly it may not be as beautiful, but eventually your users will be more appreciative of the extra mile you go into making things transparent & similar to PyTorch docs.
- xkgt 5y ago
  For people planning to learn PyTorch or even NN as a whole, it would also help to order or group the implementations in terms of complexity.
- 5y ago
rg111 5y ago
Really depends on where you are coming from. If you are already working in Deep Learning, and super comfortable in some other framework, then- -> Head over to the PyTorch website, and go through the introductory tutorials. -> Go to Papers with Code [0], and start reading and trying out implementing relatively easier research papers. If you are a beginner, then, you can go through resources that reach Deep Learning through PyTorch. I recommend- 1. https://d2l.ai https://d2l.ai 2. https://atcold.github.io/pytorch-Deep-Learning/ https://atcold.github.io/pytorch-Deep-Learning/ Although, for DL professionals wanting to learn PyTorch will get less bang for the buck from these resources. I recommend the PyTorch website tutorials for everyone, though. [0]: https://paperswithcode.com https://paperswithcode.com

Fast.ai is basically pytorch + loads of utility functions. By following the course/book, one to some extend learns what fastai does, and how it uses pytorch for practical stuff.

carbocation 5y ago
I have also had good success with FastAI as my initial hook into PyTorch. A brief post that shows just how thin the FastAI layer can be (if you want!) is here: https://muellerzr.github.io/fastblog/2021/02/14/Pytorchtofastai.html https://muellerzr.github.io/fastblog/2021/02/14/Pytorchtofas...
kriro 5y ago
I second this but once you are comfortable with fast.ai it's a good idea to reproduce some things in pure PyTorch eventually because it'll give you a deep understanding and appreciation for fast.ai as well.
willj 5y ago
One thing with fastai that annoyed me when I built a project with it was that the v1 and v2 APIs are totally different, and if you google or search stackoverflow for help with something, I found it more likely to stumble on answers for the v1 API than the v2 API. I also didn’t find their documentation super helpful for more than the most basic things (though not all documentation can be as amazing as scikit-learn).
apohn 5y ago
Am I allowed to have a contrarian opinion :) I went through the first version of FastAI (when it was Keras, torch?/tensorflow?) and forgot most of it because never did anything with Deep Learning. Then I did the FastAI V1 course again where they use FastAI library V2. I really liked the first version of the course because I felt like Jeremy did an awesome job of balancing understanding the guts of using a DL library with getting stuff done. It was a tough course, but I felt like I really understood things. I felt like the version with FastAI library V2 went too far into "Here are some commands you can use in the FastAI V2 library to to do this sexy thing with Deep Learning." I completed that course and really felt it should have been titled "A Course on the Fast AI V2 Library" I recently purchased "Deep Learning with PyTorch" by Eli Stevens. I've been working through this book and feel like it explains things a lot more. I'm haven't finished the book with it, but I do like it so far.
- carom 5y ago
  I felt similarly about the fastai course. I'm doing the deeplearning.ai course (Andrew Ng) and I really like it. Warning though, it is bottom up, so you start by implementing neural nets with numpy to understand the internals, then it gets into TensorFlow. Just another resource if people are looking to learn deep learning and not pytorch in particular.

olavgg 5y ago

There are lot of tutorials on the PyTorch website where you can start playing around with several cool examples in many different categories. https://pytorch.org/tutorials/beginner/pytorch_with_examples.html https://pytorch.org/tutorials/beginner/pytorch_with_examples...

vpj 5y ago

We did a bunch of popular research paper implementations in PyTorch with notes (annotations); might be helpful. https://nn.labml.ai https://nn.labml.ai

srvmshr 5y ago
@vpj The only drawback I see is that much of the implementation is abstracted by your helper libraries. Not everyone wants to add an extra dependency layer. Otherwise the walkthroughs are super helpful.
- deleted 5y ago
  [deleted]
- vpj 5y ago
  Agreed. But it makes implementing and testing a lot easier for us. We try to minimize the dependencies as much as possible. Do you think it also comes in the way of understanding the paper and learning how to implement it?
  - srvmshr 5y ago
    Since the backbone of the implementation is packed away in the library import, I felt it didn't quite show the code & variable interaction well enough. Don't get me wrong. It is useful & concise like you mentioned. But your target audience is beginners & adopters, and it makes it no different from another framework such as Fastai (I have major gripes with them. It has a much bottled-in experience) To be true to walkthroughs, please consider designing helper functions rather than using your framework. Admittedly it may not be as beautiful, but eventually your users will be more appreciative of the extra mile you go into making things transparent & similar to PyTorch docs.
xkgt 5y ago
For people planning to learn PyTorch or even NN as a whole, it would also help to order or group the implementations in terms of complexity.
5y ago

rg111 5y ago

Really depends on where you are coming from. If you are already working in Deep Learning, and super comfortable in some other framework, then- -> Head over to the PyTorch website, and go through the introductory tutorials. -> Go to Papers with Code [0], and start reading and trying out implementing relatively easier research papers. If you are a beginner, then, you can go through resources that reach Deep Learning through PyTorch. I recommend- 1. https://d2l.ai https://d2l.ai 2. https://atcold.github.io/pytorch-Deep-Learning/ https://atcold.github.io/pytorch-Deep-Learning/ Although, for DL professionals wanting to learn PyTorch will get less bang for the buck from these resources. I recommend the PyTorch website tutorials for everyone, though. [0]: https://paperswithcode.com https://paperswithcode.com