4 ms·

Chainer and potentially CuPy (which was extracted from Chainer to be independent) were around before PyTorch as it served as inspiration for PyTorch. I feel lik

Chainer and potentially CuPy (which was extracted from Chainer to be independent) were around before PyTorch as it served as inspiration for PyTorch. I feel like that's a good motivation for diversity in packages and ecosystems regardless of your feelings otherwise. Along with a colleague I used CuPy in first Chainer and then PyTorch for implementing the Quasi-Recurrent Neural Network (QRNN) which at the time was far faster than even NVIDIA's optimized cuDNN LSTM whilst getting the same (or better) performance for many tasks. CuPy at the time was both the easiest and most Pythonic of potential solutions for that problem - even if it did involve writing CUDA in Python strings =] n.b. Our use case was literally pushing state of the art in research - CuPy is even more Pythonic if you're hitting more standard use cases. [1]: https://github.com/salesforce/pytorch-qrnn https://github.com/salesforce/pytorch-qrnn

DanielleMolloy 8y ago
PyTorch is almost (or even literally?) a fork of chainer, which can be seen when comparing example code. The latter was much more stable than the former for quite some time after PyTorch gained big popularity through Facebook. We have been using chainer for a lot of published NN research projects and only recently moved to PyTorch because students complained that they feel they can't put the more popular framework on their CVs.. I continue to have more sympathies for chainer.
- _0ffh 8y ago
  >PyTorch is almost (or even literally?) a fork of chainer That's funny, I would have assumed PyTorch to be, like, the python version of Torch?
  - colesbury 8y ago
    The PyTorch tensor library was originally basically the Python version of Torch 7. It's now moving closer towards NumPy's API (and farther from Torch 7). Th autograd library was inspired by Chainer's design and took a lot of concepts (but not code) directly from Chainer. The neural network API is a bit of a hybrid. It's built on top of the autograd library but the layer names, implementations, and some conventions were inherited from Torch 7's NN and cuNN libraries. (EDIT: and the name "autograd" originates from HIPS autograd library, which I think predates Chainer)
Iamhisalt 8y ago
Thanks for the QRNN. What’s it like working for Socher?

DanielleMolloy 8y ago

PyTorch is almost (or even literally?) a fork of chainer, which can be seen when comparing example code. The latter was much more stable than the former for quite some time after PyTorch gained big popularity through Facebook. We have been using chainer for a lot of published NN research projects and only recently moved to PyTorch because students complained that they feel they can't put the more popular framework on their CVs.. I continue to have more sympathies for chainer.

_0ffh 8y ago
>PyTorch is almost (or even literally?) a fork of chainer That's funny, I would have assumed PyTorch to be, like, the python version of Torch?
- colesbury 8y ago
  The PyTorch tensor library was originally basically the Python version of Torch 7. It's now moving closer towards NumPy's API (and farther from Torch 7). Th autograd library was inspired by Chainer's design and took a lot of concepts (but not code) directly from Chainer. The neural network API is a bit of a hybrid. It's built on top of the autograd library but the layer names, implementations, and some conventions were inherited from Torch 7's NN and cuNN libraries. (EDIT: and the name "autograd" originates from HIPS autograd library, which I think predates Chainer)

Iamhisalt 8y ago

Thanks for the QRNN. What’s it like working for Socher?