3 ms·

Tensorflow isn't really regarded as a very welcoming open source project in the deep learning community. The deep learning academic community is already moving

Tensorflow isn't really regarded as a very welcoming open source project in the deep learning community. The deep learning academic community is already moving to PyTorch, not just because of the imperative style programming, but also to avoid Google lock-in and the pseudo-open ethos. PyTorch is used and developed by the community, and backed by multiple companies, not just one.

make3 9y ago
as a deep learning professional who enjoys tensorflow, who is surrounded by other professionals who also like tf quite a bit.. I think you are maybe exaggerating a bit here, and would like to know your sources
hoschicz 9y ago
What do you mean with 'imperative style programming'?
- jimfleming 9y ago
  Here imperative style means you can use language constructs in e.g. Python, such as if-statements and while-loops, to directly construct models. TensorFlow, Theano, and other graph-based frameworks typically require creating branches and loops as nodes in the graph. Graphs have advantages but they can be unfamiliar and sometimes difficult: Some advantages: easier to serialize the whole graph and distribute computation; optimization can also be performed across the graph nodes (e.g. see XLA in TensorFlow). Some disadvantages: it can be more difficult to write and reason about, particularly for recurrent neural networks which can utilize loops a lot; also interop with reinforcement learning environments where much of the computation is performed in an environment outside of the graph.
laingc 9y ago
I'd be interested to know how you formed this impression. My own impression is formed by: * The work I and my colleagues are doing * Recently published literature and arxiv pre-prints * Conference talks * Industry meet ups * Blog posts Based on my anecdotal experience from these sources, I see absolutely no evidence of Torch experiencing any kind of resurgence. If I had to make a call about the direction if the community as a whole, I would say that it is very clearly heading towards TensorFlow, with some holdouts using Theano and some using mxnet. Torch is used by some groups, certainly, but I have the impression that it's use is decreasing, rather than increasing.
- p1esk 9y ago
  Just to give you another anecdotal experience: I'm a researcher who used Theano for 3 years to train convnets. A couple of months ago I realized that Theano is getting too much pain to work with (main reasons being the lack of implementations for latest models, and difficulty of using multiple GPUs), so I decided to switch to a more popular framework. I looked at TF, and almost started porting my code to it, then someone suggested I look at PyTorch. After 30 minutes playing with it, I was sold. Much more intuitive. Major architectures from the last 12 months have been implemented. Dynamic graphs are probably something I will need in the future. Community is very active and helpful. The downside is that the software is not as mature as TF, and the community is smaller, but that's changing fast. p.s. What you wrote about Torch is correct though, but we are talking about PyTorch, not Torch.

make3 9y ago

as a deep learning professional who enjoys tensorflow, who is surrounded by other professionals who also like tf quite a bit.. I think you are maybe exaggerating a bit here, and would like to know your sources

hoschicz 9y ago

What do you mean with 'imperative style programming'?

jimfleming 9y ago
Here imperative style means you can use language constructs in e.g. Python, such as if-statements and while-loops, to directly construct models. TensorFlow, Theano, and other graph-based frameworks typically require creating branches and loops as nodes in the graph. Graphs have advantages but they can be unfamiliar and sometimes difficult: Some advantages: easier to serialize the whole graph and distribute computation; optimization can also be performed across the graph nodes (e.g. see XLA in TensorFlow). Some disadvantages: it can be more difficult to write and reason about, particularly for recurrent neural networks which can utilize loops a lot; also interop with reinforcement learning environments where much of the computation is performed in an environment outside of the graph.

laingc 9y ago

I'd be interested to know how you formed this impression. My own impression is formed by: * The work I and my colleagues are doing * Recently published literature and arxiv pre-prints * Conference talks * Industry meet ups * Blog posts Based on my anecdotal experience from these sources, I see absolutely no evidence of Torch experiencing any kind of resurgence. If I had to make a call about the direction if the community as a whole, I would say that it is very clearly heading towards TensorFlow, with some holdouts using Theano and some using mxnet. Torch is used by some groups, certainly, but I have the impression that it's use is decreasing, rather than increasing.

p1esk 9y ago
Just to give you another anecdotal experience: I'm a researcher who used Theano for 3 years to train convnets. A couple of months ago I realized that Theano is getting too much pain to work with (main reasons being the lack of implementations for latest models, and difficulty of using multiple GPUs), so I decided to switch to a more popular framework. I looked at TF, and almost started porting my code to it, then someone suggested I look at PyTorch. After 30 minutes playing with it, I was sold. Much more intuitive. Major architectures from the last 12 months have been implemented. Dynamic graphs are probably something I will need in the future. Community is very active and helpful. The downside is that the software is not as mature as TF, and the community is smaller, but that's changing fast. p.s. What you wrote about Torch is correct though, but we are talking about PyTorch, not Torch.