3 ms·

Anybody else has the feeling that PyTorch is to TensorFlow what Chrome was to other browsers? I started PyTorch about a month ago and was impressed how effortl

Anybody else has the feeling that PyTorch is to TensorFlow what Chrome was to other browsers? I started PyTorch about a month ago and was impressed how effortless everything was compared to TF.

davmre 9y ago
Tensorflow's advantage is that once you build your model, you can run on everything from a massive cluster to a mobile GPU without significant modification. Because you're just writing a description of a computation graph, it's easy for backend systems to process that description and optimize the execution of your model. PyTorch's imperative semantics (where the computation graph is implicitly defined at runtime by the execution of your Python code) definitely make it cleaner to do research prototyping. But AFAIK most PyTorch models need be reimplemented in lower-level code, or maybe something like Caffe2, before they can be used in production. That's a fairly significant tradeoff, which makes it hard to see PyTorch totally replacing Tensorflow anytime in the near future. That said PyTorch is obviously a great tool and it's exciting to see how it will develop and be used. (disclaimer: I work for Google, opinions are my own)
- skierscott 9y ago
  PyTorch does have ONNX, a tool to convert from PyTorch models to Caffe2 models: http://pytorch.org/tutorials/advanced/super_resolution_with_caffe2.html http://pytorch.org/tutorials/advanced/super_resolution_with_...
- JustFinishedBSG 9y ago
  > massive cluster No you can't because Google doesn't release those functions in the open source version of TF Also PyTorch will soon by able to export directly to Caffe2 / CNTK
skierscott 9y ago
I’m doing research (not deployment) and have the same feeling. PyTorch has inspired a blog post [1], Tensorflow didn’t. Briefly, the benefits of PyTorch are * easy conversion to NumPy arrays (meaning rest of Python can be used!). This is a bottleneck in Tensorflow; for reasonable sizes, PyTorch is 1000x faster. * trackbacks are easy to follow (because defines graph by running) * it’s as fast as tensorflow [2] (or at least torch is, which calls the same C functions as PyTorch, and there’s a tweet [4] by a core dev saying to expect the same speeds). Plus on the web I’ve only found anecdotes that support PyTorch faster than tensorflow. * it’s easy to extend; everything is a simple Python class. e.g., see their different optimizers [3] [1]:http://stsievert.com/blog/2017/09/07/pytorch/ http://stsievert.com/blog/2017/09/07/pytorch/ [2]:https://github.com/soumith/convnet-benchmarks https://github.com/soumith/convnet-benchmarks [3]:https://github.com/pytorch/pytorch/tree/master/torch/optim https://github.com/pytorch/pytorch/tree/master/torch/optim [4]:https://twitter.com/soumithchintala/status/835454867107897349 https://twitter.com/soumithchintala/status/83545486710789734...
jbrambleDC 9y ago
Thoughts on Keras on top of tensorflow? I have not yet committed to a deep learning framework, as up until now, I was mostly either using scikit-learn or building neural networks from scratch, straight numpy (lol) I've heard a nice thing about Keras is that it forms more of an abstraction on top of other libraries, though I could be misunderstanding.
- ssfrr 9y ago
  I don't have a lot of experience, but I recently ported an RNN model from lasagne/theano to keras/tensorflow, and the latter combination was about 4x slower. Not sure whether it's keras or tensorflow causing the slowdown. The API is nice though. I also tried to flip the keras switch to run on top of theano instead but it had issues that I didn't have time to fix, so I just stuck with the original lasagne/theano stack.
vonnik 9y ago
PyTorch adoption is growing quickly among ML researchers. TensorFlow made a splash in 2016... Jeremy Howard gives his reasons for switching to PyTorch from TF here: http://www.fast.ai/2017/09/08/introducing-pytorch-for-fastai/ http://www.fast.ai/2017/09/08/introducing-pytorch-for-fastai... Basically, some things were really hard in TF and in PyTorch they're easy. Time to insight while testing a new model is shorter.

Anybody else has the feeling that PyTorch is to TensorFlow what Chrome was to other browsers? I started PyTorch about a month ago and was impressed how effortless everything was compared to TF.

davmre 9y ago

Tensorflow's advantage is that once you build your model, you can run on everything from a massive cluster to a mobile GPU without significant modification. Because you're just writing a description of a computation graph, it's easy for backend systems to process that description and optimize the execution of your model. PyTorch's imperative semantics (where the computation graph is implicitly defined at runtime by the execution of your Python code) definitely make it cleaner to do research prototyping. But AFAIK most PyTorch models need be reimplemented in lower-level code, or maybe something like Caffe2, before they can be used in production. That's a fairly significant tradeoff, which makes it hard to see PyTorch totally replacing Tensorflow anytime in the near future. That said PyTorch is obviously a great tool and it's exciting to see how it will develop and be used. (disclaimer: I work for Google, opinions are my own)

skierscott 9y ago
PyTorch does have ONNX, a tool to convert from PyTorch models to Caffe2 models: http://pytorch.org/tutorials/advanced/super_resolution_with_caffe2.html http://pytorch.org/tutorials/advanced/super_resolution_with_...
JustFinishedBSG 9y ago
> massive cluster No you can't because Google doesn't release those functions in the open source version of TF Also PyTorch will soon by able to export directly to Caffe2 / CNTK

skierscott 9y ago

I’m doing research (not deployment) and have the same feeling. PyTorch has inspired a blog post [1], Tensorflow didn’t. Briefly, the benefits of PyTorch are * easy conversion to NumPy arrays (meaning rest of Python can be used!). This is a bottleneck in Tensorflow; for reasonable sizes, PyTorch is 1000x faster. * trackbacks are easy to follow (because defines graph by running) * it’s as fast as tensorflow [2] (or at least torch is, which calls the same C functions as PyTorch, and there’s a tweet [4] by a core dev saying to expect the same speeds). Plus on the web I’ve only found anecdotes that support PyTorch faster than tensorflow. * it’s easy to extend; everything is a simple Python class. e.g., see their different optimizers [3] [1]:http://stsievert.com/blog/2017/09/07/pytorch/ http://stsievert.com/blog/2017/09/07/pytorch/ [2]:https://github.com/soumith/convnet-benchmarks https://github.com/soumith/convnet-benchmarks [3]:https://github.com/pytorch/pytorch/tree/master/torch/optim https://github.com/pytorch/pytorch/tree/master/torch/optim [4]:https://twitter.com/soumithchintala/status/835454867107897349 https://twitter.com/soumithchintala/status/83545486710789734...

jbrambleDC 9y ago

Thoughts on Keras on top of tensorflow? I have not yet committed to a deep learning framework, as up until now, I was mostly either using scikit-learn or building neural networks from scratch, straight numpy (lol) I've heard a nice thing about Keras is that it forms more of an abstraction on top of other libraries, though I could be misunderstanding.

ssfrr 9y ago
I don't have a lot of experience, but I recently ported an RNN model from lasagne/theano to keras/tensorflow, and the latter combination was about 4x slower. Not sure whether it's keras or tensorflow causing the slowdown. The API is nice though. I also tried to flip the keras switch to run on top of theano instead but it had issues that I didn't have time to fix, so I just stuck with the original lasagne/theano stack.

vonnik 9y ago

PyTorch adoption is growing quickly among ML researchers. TensorFlow made a splash in 2016... Jeremy Howard gives his reasons for switching to PyTorch from TF here: http://www.fast.ai/2017/09/08/introducing-pytorch-for-fastai/ http://www.fast.ai/2017/09/08/introducing-pytorch-for-fastai... Basically, some things were really hard in TF and in PyTorch they're easy. Time to insight while testing a new model is shorter.