5 ms·
TensorFlow is better for deployment. Pytorch is better for research. Theano/Keras is simpler to use and a little faster than TensorFlow
by visarga 9y ago
TensorFlow is better for deployment. Pytorch is better for research. Theano/Keras is simpler to use and a little faster than TensorFlow
- kasbah 9y agoI am kind of new to all of this but as far as I understood you can use Keras with TensorFlow as well.
- MarkMMullin 9y agoYes you can - if you're new its a remarkably painless way to get started, at least compared to the pain you would otherwise endure :-)
- visarga 9y agoI remember when it took me an hour to understand what regularization is. These simple frameworks take that for granted. They are not really accessible if you don't have a good intuition about the algorithms, hyperparameters and architectures.
- curiousgal 9y ago>Pytorch is better for research. and yet a lot of researchers are using Caffe.
- DLEnthusiast 9y ago"PyTorch is better for research" is a weird, unsubstantiated statement. The fact is that few serious researchers use PyTorch (and even those complain about it). It's mostly grad students in a handful of labs. The only researchers I know who use PyTorch have been from FaceBook, and that's because they were implicitly forced to use it (PyTorch is developed by FaceBook). According to https://medium.com/@karpathy/icml-accepted-papers-institution-stats-bad8d2943f5d https://medium.com/@karpathy/icml-accepted-papers-institutio... , 3 of the top research labs in the world are DeepMind, Google Brain (and the rest of Google), and Microsoft Research. Let's see: * DeepMind: TensorFlow * Google Brain: TensorFlow * Microsoft Research: CNTK Ok, so what about academia? The top deep learning groups in academia are: * Montreal: Theano * Toronto: TensorFlow * IDSIA: TensorFlow So, what about the greater academic research community? Maybe we could get some data about who uses what by looking at the frameworks cited by researchers in their papers. Andrej did that: it's mainly TensorFlow and Caffe. https://medium.com/@karpathy/a-peek-at-trends-in-machine-learning-ab8a1085a106 https://medium.com/@karpathy/a-peek-at-trends-in-machine-lea...
- chenzhekl 9y agoFew people use PyTorch largely because it is relatively new (0.1.12). It even doesn't have distributed training capabilities (coming in 0.2). Your arguments don't say anything about frameworks themselves. It is unfair! When people say PyTorch is better for research, they mean it is more flexible, and it is easier to implement non-trivial network architectures with it, such as recursive network, which is a cumbersome task for TensorFlow. MXNet's documentation provides a good overview to these two different styles (http://mxnet.io/architecture/program_model.html http://mxnet.io/architecture/program_model.html).
- DLEnthusiast 9y agoAll of these frameworks are "relatively new". TensorFlow: 1.6 years. CNTK: 1 year. PyTorch: 0.5 year. Are they really impossible to compare? > When people say PyTorch is better for research, they mean That's not what "people" say. They tend to say the opposite. Maybe we can ask OP what he meant when he said it. > it is easier to implement non-trivial network architectures with it, such as recursive network It is interesting that you mention recursive networks. There are only a few dozens of researchers who work with recursive networks, and they are all accounted for, we know what tools they use. They use Chainer and DyNet.
- chenzhekl 9y ago> All of these frameworks are "relatively new". TensorFlow: 1.6 years. CNTK: 1 year. PyTorch: 0.5 year. Are they really impossible to compare? 1.6 years is a long time in DL community. > That's not what "people" say. They tend to say the opposite. Maybe we can ask OP what he meant when he said it. Go ahead! Ask it. > They use Chainer and DyNet. You know Chainer came far before PyTorch, which heavily influenced PyTorch's design. You are always saying XX is using XX. Why not talk about frameworks themselves? If you insisted to your idea, let's settle down. I don't want to start a framework war. I just want all frameworks to be equally considered.
- pilooch 9y agoWith all due respect you don't appear to know much about what you are commenting about. There is a huge community of research and applications around rnn and all other architectures. Pytorch has an extremely vivid and fast growing codebase, across all neural architectures and applications, it's remarkable actually. One reason might be because it is simple an effective, including easy debugging. Another is because research now is much focused on new possibly complex and exotic architectures, new optimizers, inner guts and behavior understanding, including theoretically. And it appears pytorch makes that easy. Don't throw it away too fast as a DL enthusiast :)