4 ms·
Many of the comments here are judging PyTorch in hindsight, which is unfair. When Soumith Chintala co-launched PyTorch, and for many years after, the alternati
by cs702 11mo ago
Many of the comments here are judging PyTorch in hindsight, which is unfair.
When Soumith Chintala co-launched PyTorch, and for many years after, the alternatives for fast, interactive, convenient development were much worse. There was no Jax.
Every single AI researcher I know, including me, who tried PyTorch back then immediately wanted to switch to it, because it was so much better. Andrej Karpathy described what PyTorch felt like back then when he tweeted, on May 2017, "I've been using PyTorch a few months now and I've never felt better. I have more energy. My skin is clearer. My eyesight has improved."[a]
THANK YOU SOUMITH for your hard work over all these years! Your hard work has made a difference for a huge number of people, including many of us here on HN.
We wish you success in your future endeavors, whatever they turn out to be!
Please ignore all the petty criticism.
---
[a] https://x.com/karpathy/status/868178954032513024 https://x.com/karpathy/status/868178954032513024
- golly_ned 11mo agoThere was Chainer, which originated the define-by-run model that characterized PyTorch’s effectiveness. It was developed by a much smaller, much less influential company in Japan. Early PyTorch is transparent about the debt owed to Chainer.
- maxc01 11mo agoYes, exactly—not many people know about Chainer nowadays. Back in 2016, PyTorch's interface was actually inferior to Chainer's, and I think Chainer's design was really ahead of its time.
- NalNezumi 11mo agoThe company is called preferred networks, and they're still around, and have some branched off subsidiaries too.
- cs702 11mo agoThanks. Yes, I remember Chainer, but only vaguely. I kinda remember looking at it, but not actually using it. My recollection is that when I looked at Chainer back then, it didn't offer a comprehensive library of preexisting components for deep learning. When I tried PyTorch, on the other hand, I vividly remember it as already having lots of prebuilt components (common layers, activation functions, etc.) in `torch.nn`, so it was easier and faster to get going. These memories are vague, so I could be wrong.
- Teodolfo 11mo agoPyTorch was partly inspired by the python Autograd library (circa 2015 [1]) to the point where they called their autodiff [2] system "autograd" [3]. Jax is the direct successor of the Autograd library and several of the Autograd developers work on Jax to this day. Of course, for that matter, PyTorch author Adam Paszke is currently on the JAX team and seems to work on JAX and Dex these days. [1] https://pypi.org/project/autograd/#history https://pypi.org/project/autograd/#history [2] https://www.cs.toronto.edu/~rgrosse/courses/csc421_2019/readings/L06%20Automatic%20Differentiation.pdf https://www.cs.toronto.edu/~rgrosse/courses/csc421_2019/read... [2] https://web.archive.org/web/20170422051747/http://pytorch.org/about/ https://web.archive.org/web/20170422051747/http://pytorch.or...
- cs702 11mo agoYes, PyTorch borrowed from Autograd, Chainer, etc. ...but PyTorch felt friendlier and more Pythonic, and it came with a comprehensive library of prebuilt components for deep learning in `torch.nn`. See https://news.ycombinator.com/item?id=45848768 https://news.ycombinator.com/item?id=45848768
- make3 11mo agoI don't know why you assume that people are saying negative things about PyTorch. Most people by very very far really love it.