5 ms·

Will this release change the landscape of deep learning frameworks and lessen the Tensorflow lead? In other words, for someone that has not touched DL before, i

Will this release change the landscape of deep learning frameworks and lessen the Tensorflow lead? In other words, for someone that has not touched DL before, is it worth starting with PyTorch instead of TF?

deleted 8y ago
[deleted]
deleted 8y ago
[deleted]
Q6T46nT668w6i3m 8y ago
I don’t know. I use both. I contribute to both. If you’re interested in supervised neural networks, Keras, in my opinion, is the best option. You’d use CNTK, or more likely, TensorFlow for your Keras backend. If you’re interested in unsupervised or semi-supervised neural networks, TensorFlow and PyTorch both work. However, they both have noteworthy issues. Until eager execution was added to TensorFlow, TensorFlow models needed to be compiled. It was difficult to spice your network with dynamic behavior that couldn’t be easily constrained to a TensorFlow graph. Especially if you weren’t strong programming with common parallel primitives. Eager execution has made this easier, but it’s still more cumbersome than PyTorch’s execution methodology. PyTorch sacrifices many of the benefits of compilation for usability and this is most obvious when deploying to, for example, the cloud or mobile devices. PyTorch, also, inexplicably, breaks many of the conventions present in the scientific Python ecosystem making it pretty cumbersome to integrate into existing workflows that rely on a package like scikit-image. It’s complicated. In fact, I believe it’s way too complicated. Thankfully Keras solves most problems for most people in this area.
- thecage411 8y ago
  What are the scientific Python ecosystem conventions that PyTorch breaks?
  - Q6T46nT668w6i3m 8y ago
    The most obvious is API compatibility (e.g. function definitions) with NumPy, SciPy, scikit-image, etc.
- buildbot 8y ago
  That breaking of compatibility is why I often recommend Chainer [1] over Pytorch, as one of the back ends to Chainer is literally numpy. Plus, Chainer is very fast and easy to work with. [1] https://chainer.org https://chainer.org
  - Q6T46nT668w6i3m 8y ago
    Yes! It was a mistake not to mention Chainer since it’s an exceptional package. Honestly, it presently fulfills the PyTorch mission better than PyTorch!
make3 8y ago
once you know one, learning the other is very easy. I'd start with pytorch as it's much easier to reason about and debug
stared 8y ago
I wrote a whole article about that: "Keras or PyTorch as your first deep learning framework" (https://deepsense.ai/keras-or-pytorch/ https://deepsense.ai/keras-or-pytorch/). And it got somewhat popular here: https://news.ycombinator.com/item?id=17415321 https://news.ycombinator.com/item?id=17415321 (if you are curious of others' opinions). In short: don't learn (raw) TensorFlow as your first framework. Both TF-via-Keras and PyTorch are viable options, with their pros and cons.
nafizh 8y ago
Pytorch is way easier to learn DL than Tensorflow because of its natural place within the python workflow. Just write a simple 1 layer feed-forward NN both in pytorch and TF, and you will see the difference.
_untom_ 8y ago
What's your source for Tensorflow being in the lead? I'd love to have some actual data on this. In my experience, for a long time now, I've barely seen anyone outside Alphabet doing anything with TF. Everywhere I look and everyone I talk to uses PyTorch. So from my perspective, PyTorch is (IMO deservedly) miles ahead in usage, and TF 2.0 will probably do only little to anything to close that gap. Disclaimer: I'm in research, things might be different in production environments.
- Q6T46nT668w6i3m 8y ago
  I don’t know your area of expertise (or familiarity) but TensorFlow is extremely present in both academic and industrial computer vision communities. I don’t have data to support my claim, but, from my experience, TensorFlow is likely the most popular framework. Hell, I’d bet Caffe (1) is still more popular than PyTorch (but I’d expect that to change)!
  - _untom_ 8y ago
    I'm in Deep Learning research (e.g., you'd see me at NIPS, ICML or NIPS), so I don't have too much insight into the CV community. Thanks for sharing, interesting to know that they still rely on TF so much. As far as deep learning research goes, PyTorch is the clear winner by now. At least that's my subjective impression, but I also lack data. One recent data point is this one about frameworks used for ICLR 2019 submissions compared to the previous year: https://www.reddit.com/r/MachineLearning/comments/9kys38/r_frameworks_mentioned_iclr_20182019_tensorflow/ https://www.reddit.com/r/MachineLearning/comments/9kys38/r_f... , which would suggest about parity in terms of usage: TensorFlow went from 228 mentions last year to 266 this year, Pytorch from 87 to 252. Considering that a large number of these submissions is from Alphabet (this NIPS 10% of the papers were from either Google or DeepMind, so ICLR will be similar), so they naturally use TF, meaning that PyTorch actually as a slight lead (actually less than expected, but still. At least the growth numbers are quite impressive).

deleted 8y ago

[deleted]

deleted 8y ago

[deleted]

Q6T46nT668w6i3m 8y ago

I don’t know. I use both. I contribute to both. If you’re interested in supervised neural networks, Keras, in my opinion, is the best option. You’d use CNTK, or more likely, TensorFlow for your Keras backend. If you’re interested in unsupervised or semi-supervised neural networks, TensorFlow and PyTorch both work. However, they both have noteworthy issues. Until eager execution was added to TensorFlow, TensorFlow models needed to be compiled. It was difficult to spice your network with dynamic behavior that couldn’t be easily constrained to a TensorFlow graph. Especially if you weren’t strong programming with common parallel primitives. Eager execution has made this easier, but it’s still more cumbersome than PyTorch’s execution methodology. PyTorch sacrifices many of the benefits of compilation for usability and this is most obvious when deploying to, for example, the cloud or mobile devices. PyTorch, also, inexplicably, breaks many of the conventions present in the scientific Python ecosystem making it pretty cumbersome to integrate into existing workflows that rely on a package like scikit-image. It’s complicated. In fact, I believe it’s way too complicated. Thankfully Keras solves most problems for most people in this area.

thecage411 8y ago
What are the scientific Python ecosystem conventions that PyTorch breaks?
- Q6T46nT668w6i3m 8y ago
  The most obvious is API compatibility (e.g. function definitions) with NumPy, SciPy, scikit-image, etc.
buildbot 8y ago
That breaking of compatibility is why I often recommend Chainer [1] over Pytorch, as one of the back ends to Chainer is literally numpy. Plus, Chainer is very fast and easy to work with. [1] https://chainer.org https://chainer.org
- Q6T46nT668w6i3m 8y ago
  Yes! It was a mistake not to mention Chainer since it’s an exceptional package. Honestly, it presently fulfills the PyTorch mission better than PyTorch!

make3 8y ago

once you know one, learning the other is very easy. I'd start with pytorch as it's much easier to reason about and debug

stared 8y ago

I wrote a whole article about that: "Keras or PyTorch as your first deep learning framework" (https://deepsense.ai/keras-or-pytorch/ https://deepsense.ai/keras-or-pytorch/). And it got somewhat popular here: https://news.ycombinator.com/item?id=17415321 https://news.ycombinator.com/item?id=17415321 (if you are curious of others' opinions). In short: don't learn (raw) TensorFlow as your first framework. Both TF-via-Keras and PyTorch are viable options, with their pros and cons.

nafizh 8y ago

Pytorch is way easier to learn DL than Tensorflow because of its natural place within the python workflow. Just write a simple 1 layer feed-forward NN both in pytorch and TF, and you will see the difference.

_untom_ 8y ago

What's your source for Tensorflow being in the lead? I'd love to have some actual data on this. In my experience, for a long time now, I've barely seen anyone outside Alphabet doing anything with TF. Everywhere I look and everyone I talk to uses PyTorch. So from my perspective, PyTorch is (IMO deservedly) miles ahead in usage, and TF 2.0 will probably do only little to anything to close that gap. Disclaimer: I'm in research, things might be different in production environments.

Q6T46nT668w6i3m 8y ago
I don’t know your area of expertise (or familiarity) but TensorFlow is extremely present in both academic and industrial computer vision communities. I don’t have data to support my claim, but, from my experience, TensorFlow is likely the most popular framework. Hell, I’d bet Caffe (1) is still more popular than PyTorch (but I’d expect that to change)!
- _untom_ 8y ago
  I'm in Deep Learning research (e.g., you'd see me at NIPS, ICML or NIPS), so I don't have too much insight into the CV community. Thanks for sharing, interesting to know that they still rely on TF so much. As far as deep learning research goes, PyTorch is the clear winner by now. At least that's my subjective impression, but I also lack data. One recent data point is this one about frameworks used for ICLR 2019 submissions compared to the previous year: https://www.reddit.com/r/MachineLearning/comments/9kys38/r_frameworks_mentioned_iclr_20182019_tensorflow/ https://www.reddit.com/r/MachineLearning/comments/9kys38/r_f... , which would suggest about parity in terms of usage: TensorFlow went from 228 mentions last year to 266 this year, Pytorch from 87 to 252. Considering that a large number of these submissions is from Alphabet (this NIPS 10% of the papers were from either Google or DeepMind, so ICLR will be similar), so they naturally use TF, meaning that PyTorch actually as a slight lead (actually less than expected, but still. At least the growth numbers are quite impressive).